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PREFACE 


IN this book an account is given of an enquiry, extending 
over several years, into the reliability of examinations so 
far as this can be tested by the performance in subsequent 
yeats of those examined. In particular the selection of 
pupils for secondary schools and the award of scholarships 
at the universities are dealt with. 

It should be made clear at once that this is no titade 
against examinations in general. They seem to the writer 
an inevitable part of out educational system at some stages 
and for some purposes. What is urgent is that they should 
not dominate, and that we should know where their 
strength and weaknesses lie. An important aspect of this 
general problem is, how far we can rely on examination 
results, and that is our main topic hete. 

It should also be emphasised that this is no attack upon 
examining bodies in general. My prejudices are indeed 
likely to be in favour of such examining bodies ; for I am 
fortunate in having had the opportunity of seeing the 
planning, development, and managing of such examinations 
from the inside, both as a member for twelve years of the 
Joint Northern Matriculation Board, and as first Chairman 
of the Examining Board to establish and control a general 
examination for the admission of pupils to, and the award 
of free places for, the Grammar Schools and the Municipal 
Secondary Schools in the City of Birmingham. 

In both those bodies I have seen the рег? 
almost meticulous саге taken by officials ~ 
of schools, and members of boards tc ` 
and fairness of the examinations ^ 
doubt that the majority of exa* 
difficult tasks with a like ^^ 
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to use every kind of examination technique, every device, 
statistical and psychological which may improve the 
reliability and fairness of their examinations. 

An essential preliminary to the accomplishment of such 
aims is to know the facts. And it is to present some of 
these facts, however disturbing they may be, that this 
report is published. 

It is, of course, impossible to deal with a subject like this 
without the presentation of figures and statistics. I hope, 
however, that those to whom even elementary mathematics 
ate unpalatable will not be alarmed. I have had such 
readers definitely in mind, and I think throughout the treat- 
ment is as simple as it could be made. Detailed figures and 
the special treatment of some mathematical points ate 
collected in the Appendix, to which explicit references in 
the text will guide those interested. І may add that after 
Part I either Part П or Part III may be read first. 

This enquiry would not have been possible but for the 
kind co-operation of many helpers. My chief debt is to 
my friend and colleague Mr. W. G. Emmett, who has done 
for me nearly all the detailed statistical work and frequently 
helped me with valuable suggestions. 

My watm thanks are also due to my own Vice- 
Chancellor, Sir Charles Grant Robertson, for his kindness 
in reading the proofs of the chapters relating to university 
scholarships and of Part I, and for making some most use- 

"observations; to Sir Michael Sadler for reading the 
* of the chapters on university scholarships, and 
--outaging counsel at an earlier stage of 

Udny Yule, of St. John’s College, 

^l reading of the same chapters 

view; to Dr. P. D. Innes, 

“ity of Birmingham, and 
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to Mr. H. P. Lunn, Headmaster of Yardley School, 
Birmingham, for teading the chapters on the secondaty 
school entrance examination, and for many helpful 
comments; to Professor Godfrey Thomson and Mr. 
Frank Sandon for a critical reading of the chapters on the 
schools from a statistical point of view, and for their 
valuable opinions on some important points, and to Miss 
Matjorie Lewis for her patient interpretation of a difficult 
manuscript. 

My thanks ate also due to a number of others—Heads 
and Tutors of various colleges at Oxford and Cambridge, 
the Registrars of several provincial universities, Directors 
of Education and Heads of secondary schools, who have 
sent me particulars, and patiently answeted many questions 
on points of detail ; these, however, I must not name, as it 
is desirable that the various modern universities, the parti- 
cular colleges at Oxfotd and Cambridge, and also the city 
and county centres for the entrance examinations to 
secondary schools should not be indicated. I wish further 
to acknowledge the courtesy of the Board of Education in 
supplying me with very full information as to the perform- 
ance of State Scholars at the universities, and the receipt of 
a very useful grant from the Research Committee of the 
University of Birmingham for the expenses of this enquiry. 

C. W. VALENTINE. 


UNIVERSITY OF BIRMINGHAM. 
1932 
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OnE of the main causes of the growing concern about 
examinations is the harmful influence they are thought to 
have on the work of the schools—in spite of the value to 
many pupils of the stimulus provided by an examination 
ahead. However carefully examining bodies may try to 
adapt syllabuses and regulations to the needs and capacities 
of pupils at different stages, it is argued that they cannot 
do this so well as those who are in living contact with the 
pupils themselves. In secondary schools the influence of 
examinations is especially felt through matriculation regula- 
tions. Subjects reasonably required for further studies at 
the univetsity are imposed, through the practical unification 
of matriculation and School Certificate examinations, upon 
nearly all secondary school pupils, the great majority of 
whom will never go to a university. 

Further, examinations can only deal with the dry bones 
of some subjects. The main aim of the study of literature, 
for example, is surely to cultivate appreciation. It is difi- 
cult, however, to devise examination questions which can 
do more than test a knowledge of facts about the literature ; 
and the danger is lest the need to prepare for the examina- 
tion should warp the whole study of the subject, though 
admittedly things are much better than they were in the 
writet’s own school-days, when it was more important for 
examination purposes to tead the introduction to a play of 
Shakespeare and the notes than it was to read the play itself. 

Also, the greater facility in examining on certain parts of 
a subject is apt to emphasise those parts in the examination 
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and consequently in the work of the school. Consider the 
study of a modern language. There is an increasing belief 
that for the average pupil wider reading in the language 
(for comprehension and appreciation) is important, even 
at the sacrifice of practice in composition.? 

Yet the difficulty of readily discriminating between pupils 
by their translation from French into English is much 
greater than it is if composition and grammar questions are 
set. In the latter case there are far more mistakes to the 
page; whereas it would be necessary to give several pages 
of French for translation to discriminate equally well 
between the briliant and the good, the good and the 
average, the.average and the weak. 

Again, there is the influence on school work of the mere 
inclusion or exclusion of a subject in the examination 
syllabus. I well remember the difficulty I felt in voting as 
to whether music should be included as a subject fot a 
certain School Certificate and matriculation examination. 
I was anxious to encourage the appreciation and serious 
study of music in the schools; yet I feared the cramping 
effect of preparing pupils for a written examination about 
music. Even the very best schools, holding to high ideals 
of education, can hardly fail to be influenced by examination 
syllabuses and regulations. In the worst type of school the 
effects may be disastrous. One of my students, for ex- 
ample, told me that in her own school the pupils studied 
the Hanoverian period in English bistoty for four consecu- 
tive years (instead of reading all the periods leading up to 
it first), because that was to be the period selected for their 
School Certificate examination. 

Even in schools of considerable repute, where such 


2 See the report of the Committee on the Position of French in the School Certifi- 
cate Examination, especially p. 9 (Board of Education Pamphlets, No. 7o, 1929). 
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methods would be repudiated, the curricula of pupils are 
sometimes unduly restricted so that they may concentrate on 
the five essential subjects required for the School Certificate. 

‘At university stage the cramping influence of examina- 
tions is fortunately not nearly so great, yet even hete it is 
not altogether absent. A professor of mathematics, for 
example, recently protested against the time often wasted 
at Cambridge by the working of many examples, in order 
to turn an easy second-class honours man into a just first- 
class, though the time would be far more profitably spent 
on wide reading and general principles. As a sidelight on 
the possibility of independence of thought being discour- 
aged by examinations, I may quote a recent testimonial 
from a distinguished university professor of English, who, 
after commending the originality of a former student of his, 
wrote: “Indeed, to begin with, Miss X sometimes failed 
to do herself full justice, because she was unwilling to sub- 
stitute for her own thought a reproduction of the thoughts 
of others.” In general, however, such a danger is not, I 
imagine, great. It is so refreshing (and rare) to find 
independence and originality of thought in a student’s 
work that one gladly rewards it. 

Much more might be said on the question of the influence 
of examinations on education, but it is not the main problem 
of this book. It is relevant only because, so far as the 
influence of examinations may be educationally harmful, it 
is all the more important to enquire how far they are 7e- 
liable. A revelation of great unreliability should tend to 
lessen the weight attached to examinations and so decrease 
their dominance in the general scheme of education." 


1 ‘The unrest among teachers, due to the fear of the harmful influence of examina- 
tions, is vigorously expressed by Dr. Cyril Norwood in his arresting book, The 
English Tradition of Education, London, 1929 (chap. xiii). 
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This question as to the reliability of examinations is 
made mote important because a vast network of mote ot 
less standardised examinations is spread over practically the 
whole educational system. One net covers the elementary- 
school pupils who try to pass through into the secondary 
school; another is over those who seek to leave the 
secondaty school with a School Certificate, ot who wish to 
enter a university ; another with a smaller mesh for those 
seeking scholarships at the modern universities; and yet 
another for those seeking scholarships at one of the ancient 
universities. 

Many people ate wondering whether these various tests 
always select the best pupils, especially those pupils who 
are most capable of profiting from the further education, 
Some teachets, both school and university, ate convinced 
they often do not. 

Opinions, however, even of experienced teachers, ate 
very conflicting, and before we can form confident judg- 
ments we need more facts. It was with a view to discover- 
ing some important facts in reference to examinations that 
'the present enquiry was undertaken. 

First, however, it may be well to examine briefly the 
reasons for the distrust felt by many as to the reliability of 
examination awards. 


SUPPOSED CAUSES OF THE UNRELIABILITY OF 
EXAMINATIONS 


(а) In the first place some feel that the element of шк? 
in the type of questions a candidate happens to meet has 
still tóo much weight. Naturally this is an important 
factor in the view of the candidate who has failed. So often 
* the questions did not suit him.” For the fully prepared 


Chap. 1.] * LUCK IN EXAMINATIONS I9 


candidate, however, the one who has carefully covered the 
whole of the prescribed course, this aspect of luck is of little 
significance, and examiners familiar with the technique of 
examinations bear in mind the danger that the question . 
paper may ignore any important sections of the syllabus 
ot deal too fully with one patt only. 

Yet the element of luck cannot be entirely disregarded, 
and may be decisive in border-line cases. It works both 
ways of course. Every teacher knows cases in which 
exceptionally lucky choice in revision immediately before 
.an examination has just pulled a weak candidate over the 
border. On the other hand, a particular concatenation of 
questions on a subject in which the paper cannot cover all 
the work (e.g. in history) may happen not to ‘suit’ a pat- 
ticular pupil or even a whole class. Such seems to be the 
explanation of a curious result shown in a certain School 
Certificate examination in history. An excellent school 
had had remarkable results in history, with many high dis- 
tinctions and 98 per cent. of passes over a period of five 
years. Then one year, with the same master in charge of 
the class, and with boys who in their other work appeared 
-of similar calibre to those of earlier years, there was a 
sudden, drop in distinctions and credits and a large increase 
in failures. The teacher and the headmaster thought that 
some ertor had been made in the returns, but enquiry did 
not support this.? 

The element of luck due to the selection of questions is 
much less in some subjects than in others. It should have 
little influence in a thorough examination in a language, if 
the ground is adequately covered. In mathematics and 


1 The indignant teacher indicated, as the sole change he knew of concerning 
history teaching in the neighbourhood, the fact that a branch of the Historical 
Association had been founded in the town that year l 
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some other subjects it can be met at an elementaty stage by 
the device suggested by Dr. Ballard, by which many vety 
short questions ate set covering the whole of the work,1 and 
it is met along somewhat similar lines by the plan of the 
papers set by the Joint Northern Matriculation Board in 
mathematics and to some extent in chemistry. 

Variability in the relative difficulty of alternative ques- 
tions is another disturbing factor. Even teachers engaged 
in day to day work with pupils are not infallible in their 
judgment as to what questions will be hard or easy for their 
own pupils? Such errors of judgment are much more 
likely to occur among examiners in public examinations ; 
and the hurried selection by a candidate of one ot 
two questions unsuspectedly difficult may penalise him 
sevetely. 

Further, speed of working is an important factor in 
success in long papers, which may unduly hamper some 
pupils, whereas short papers give no extra credit to more 
intelligent candidates who have done their work long 
before duller but plodding pupils. 

An aspect of luck which has less significance for the great 
majority is the mental condition or ‘form’ the candidate 
seems to be in on the day of examination. With the nervy 
‘highly strung’ candidate this may be important. And 
* bad luck in the working of his first question may cause 
him to be erratic in the rest of the paper. No doubt this 
variability of form, or exceptional ‘ bad luck ° in the ques- 
tions set, or unwise choice by the candidate as to which 
questions he will attempt, are the main reasons why one 
hears so often an internal examiner say that so-and-so has 
“ failed to do himself justice." 


1 See P. B. Ballard, The New Examiner, London, 1923. 
* See Measuring the Results af Teaching, by W. S. Monroe, р. 14. 
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Mote than once, when acting as External Examiner, have 
I heard university teachers say with a sigh that a certain 
student must fail because of one markedly weak paper, 
though they declared him to be one of the best students, 
from their knowledge of his work during the year. It was 
the absurdity of such recurrent situations that impelled me 
to institute a further viva-voce test in my own examinations 
at Birmingham University for students on the border-line 
in their written work—either border-line ‘ fails? or border- 
line ‘ firsts? Such supplementary viva-voce tests, however, 
are not possible in large examinations like those for the 
School Certificate, in which thousands of candidates are 
involved. 

There is one other aspect of ‘ luck * which may be serious 
but which can hardly be countered. It is difficult to esti- 
mate the disadvantage under which some adolescent girls 
suffer if a critical paper coincides with a time of unusual 
pain or nervous depression. 

All these aspects of ‘luck’ are probably factors in pto- 
ducing varying results in different examinations in the same sub- 
ject about the same time, an aspect of unreliability to which I 
shall refer as instability, and as to which I shall give some 
evidente later. 

(b) A second ground for uneasiness felt as to the relia- 
bility of examination awards is the apparently adventitious 
nature of some decisions as to a pass or fail. Thus in the 
School Certificate examination or matriculation a few marks 
may make the difference between success and failure in one 
subject, and that subject may be essential for the award of a 
certificate. For example, in Fig. 1 below are shown the 
matks of two boys in a secondary-school examination. 
Boy A, with two distinctions and one subject marked good, 
and with a much greater total of marks than В, fails to get a 
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certifieate because of weakness in French, though he 
gets only a few marks less than B in French, who 
obtains the certificate. All who know the variability of 
examination papers, or of the performance of pupils, 
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know well that at another examination in French on the 
next day—or even on the same paper—the position of B 
and A might be reversed. It is possible that they might 
be reversed even if the same scripts were handed to a 
different examiner. 

Another boy at the same school obtained three.‘ distinc- 
tions’ and had two subjects marked ‘ good’ (which is 
higher than ‘ credit °), but failed to get a School Certificate 
(which only requires five passes and one credit). These are 
only two examples from one school in one or two years. 
" Many of us," writes the Headmaster of Holt School, 
Liverpool, * have had the heart-breaking experience of a 
candidate, the best in his year, with as many as four distinc- 
tions, failing to get matriculation or even a School Certifi- 
cate.” Such cases seem to occur constantly. Nor can 
one blame the Examination Boards. ‘They must fix a limit 
of marks somewhere ; and even with the principle of com- 
pensation for weakness in one subject by average strength, 
hard cases will appeat at the margin? 


1 © Reform of Examinations in Secondary Schools in England,” in the Educa- 
tional Outlook (University of Pennsylvania Press). 

? Tt is not part of the present enquiry to discuss the desirability of certain com- 
pulsoty subjects being required for a pass in the School Certificate examination. 
But evidently, the hardships of the type given (if they are so regarded) would be 
lessened by an extended sliding scale of compensation, so that even great weakness 
in a compulsory subject could be compensated by sufficiently high marks in other 
subjects. In some examinations this system is introduced to some extent in that 
one compulsory gtoup may be passed at a slightly lower standard (provided there 
is adequate compensation in other subjects). To carry the principle much farther 
would in effect be to give up the compulsory group system for those pupils suffici- 
ently distinguished in other groups. And few educationists wish to encour” 
further specialisation at so early a stage. On the other hand, many teachers cc 
that some subjects which are essential for the award of the School Certificat 
not be so. Thus, at a large conference of Seccndary School Headm: 
Headmisttesses of Birmingham and the Midlands, general applause fo 
statement by one headmaster to the effect that many boys, quite go 
respects at school, were unable to pass the School Certificate in a forei 
just as probably there were many girls who could not pass in mathe 
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In the same school as that already referred to, we 
find the following extraordinary results in one recent 
Сав: 
: 15 boy, with 1,328 marks, had gained School Certifi- 
cate and matriculation. 
2nd boy, with 1,307 marks, had not gained matricula- 
tion or even School Certificate. 
3td boy, with 1,303 marks, had gained School Certifi- 
cate and matriculation. 

Pupils numbered 4th to zoth all gained the School 
Certificate, though No. 20, the bottom of the class, only 
obtained a total of 877 marks, against a total of 1,307 
gained by the znd boy, who failed to get a Certificate, 
through failing—as did the boy in Fig. I—to satisfy the 
examiners in one of the compulsory groups.! 

The arbitrary nature of some awatds is felt especially 
in reference to matriculation certificates. For a boy who 
carefully distributes his work over the five essential subjects 
(in which a © credit * must be obtained) may gain a matricula- 
tion certificate, though he may do worse in the examination 
as a whole than does another boy who misses by a mark or 
two a credit in one of the essential subjects. 

Such possibilities make all the more deplorabie the 
extreme reverence some people have for the word ‘ matricu- 
lation ’ as a clue to the type of result gained in the School 
Certificate examination. The head of the school from . 
which the above results are givetr told me that a firm of 
“ccountants asked him for а good boy for their office. Не 

"mended A, who had got distinction in the School 
‘cate in both mathematics and English. But A had 
the magic title matriculation, and the accountants 
‘hod of obtaining the ‘ total’ marks was the same as that described in 

р. 52. In this cass every pupil took at least seven subjects. 
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insisted on taking one who had, and they chose B—who 
had зоо marks less than A on the total. 


1 Such marked individual anomalies are of course quite consistent with there 
being a general tendency among large numbers of pupils for the ablest to gain the 
matriculation standard and not merely satisfy requirements for the School Certifi- 
cate. It may also be that the matriculation requirements do place a particular 
demand upon general ability (as contrasted with specific abilities). For this some 
slight evidence will appear later, and will be discussed in Appendix I (4), p. 176. 


Chapter II 


THE INSTABILITY OF THE MARKING OF EXAMINATION 
ANSWERS, AND OTHER CRITICISMS OF EXAMINATIONS 


In the previous chapter we have given several reasonable 
grounds for the distrust of the reliability of examination 
awards. 

We must consider with greater detail а third and more 
serious ground for lack of confidence in examination awards, 
namely, personal variability and unreliability of marking 
in the examiners themselves. This is not great in examina- 
tions in such a subject as elementary arithmetic, though 
even here individual examiners may vary in the marks 
they would allow for ‘attempts’ and the use of the best 
ot inferior methods. It may, however, be very great in 
English and history, and to a less extent in all other 
subjects in which the factor of composition enters.! 

In the marking of essays especially, extraordinary varia- 
tions occur between the marks of different examiners ; and 
this is most important in view of the prominent place that 
the essay occupies in many examinations for entrance to 
secondary schools ; for example, in one of the eight centres 
I have dealt with in Chapters III and IV, the English’ paper 
consisted entirely of two short essays. For the School 
Certificate examination a pass in English is usually com- 
pulsory ; and for some University Scholarships the essay 
paper is important, and other quéstions ate often of the 


essay type. 


* On the common influence of the factor of composition in humanistic and 
science subjects, see Professor Burt’s report on The Distribution and Relations of 
Educational Abilities (issued by the London County Council, 1917). Ata meeting 
of the Science Masters’ Association, in January 1926, Mr. D. Bertidge, of Malvern 
College, expressed the opinion that elementary examinations, even in practical 
science, tended to become a test of the candidate’s power to ‘ write an essay.’ 


26 
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In view of some striking statements about the untelia- 
bility of essay marking, I gave a test in the uniformity of 
essay marking to a group of teachers who were taking my 
class for the Masters’ degree in Education (Session 1924-5). 
The topic of the essay was a simple one—“ Town versus 
Country Life ”—and the pupils who had written the essays 
were II to 12 years of age. There were seventeen essays. 
The teachers were asked to mark with special care, and they 
knew it was a test of their own reliability in marking. A 
maximum of ten marks was to be awarded for ideas, and 
ten for expression ; spelling was to be ignored. 

One essay was outstanding, and was placed first by ten 
out of thirteen markers. One essay was placed last by 
seven matkers. But all the intermediate essays had most 
varied places assigned. Thus, the essay of E. S. was placed 
second by one examiner, third by two others, but sixteenth 
(out of seventeen) by another, and thirteenth by another. 
The essay of W. S. was placed first by one marker, 
sixteenth by another, and at vatious intermediate places 
by others. І 

Suppose we divide the awards into three groups accotd- 
ing to order of merit; thus: 

Nos 1-6. Pass first division. 

Nos. 7-11. Pass second division. 

Nos. 12-17. Fail. 

Then of the seventeen pupils, eleven wete given a first 
division pass by some examiners, and a fail by others. 

Of course, іп the large public examinations a team of 
examiners would be under a chief examiner, whose business 
it is to check the standards of the team. But all the,essays 
cannot be re-marked, and the standard of one marker him- 
self varies from time to time. It is extraordinarily difficult 
to keep the same standard of marking through a long series 
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of papets; and the re-marking of the fifty ot so first 
marked is no guarantee of consistency throughout.!. 

There is a further possibility of error in the influence of 
suggestion on a chief examiner, who in checking the 
marks of another is already aware of the marks previously 
given. Some examining boards ate aware of this danger 
of suggestion, but it is uncertain how far it is generally 
guatded against.? | 

The gteat difficulty of standardising the marking of the 
essays no doubt accounts for some of the anomalies one 
hears of at times ; as, for example, the instances of pupils 
who gained distinction in English literatute in the School 
Certificate, but failed in English composition! Such cases 
may be rate ; but one school reports seven examples, within 
four yeats, of failure in ‘composition’ together with 
‘credit’ in English literature. 

The vatiations in estimates of style and other values also, 
no doubt, account for the fact that the public examination 

1 The unteliability of essay marking is dealt with trenchantly by Dr. Ballard 
in his New Examiner. Exact correlations between the orders of merit given by 
different examiners of essays ate given by Godfrey Thomson and Stella Bailes, in the 
Forum of Education, vol. iv, 1926, p. 85. In his book, The Technique of Examining 
Children (London, 1930), Mr. B. C. Wallis describes a scheme for the marking of the 
essays of young children which seems to have worked well in several latgg centres, 
Marks are given for seven aspects of the essay (Vocabulary, Accutacy, Quality of 
Phrases, Consistency, Quantity of Ideas, etc). It would certainly seem possible 
by some such method to avoid great variability in one’s own standard, judging 
by a recent enquiry of one of my research students, Mr. О, J. Cotton. He re- 
marked six sets of essays after three months, and thus obtained two orders of merit. 
The marks for two sets of descriptive essays (added together) gave, on re-marking, 
an order correlating with the first marking to an extent of 0:95, ina class of 31 boys, 
ages 14-16, Imaginative essays gave a correlation of 0-93 (33 boys, ages 12-14). 
Two sets of narrative essays gave a correlation of o:95 (25 boys, ages 10-12). (The 
significance of such correlation figures is explained on p. 57. ‘The adding of the 
marks of two essays would, however, probably steady the orders.) 

* The danger is not ignored by the Joint Northern Matriculation Board authori- 
ties, See the admirable exposition of their methods given іп Secondary School 


Examination Statistics, by J. M. Crofts and D. Caradog Jones (London, 1928), 
especially p. 42. 
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results of school pupils in such * composition subjects as 
English and history differ more from the school estimates 
of the pupils than do the examination results in mathe- 
matics and French (while the School Certificate order in 
English composition differs still more widely from the 
order of merit drawn up by the schools): 

Now as just mentioned, this difficulty in standardising 
the marking of essays is especially serious in view of their 
critical importance in some examinations. Undoubtedly 
there are good reasons for such importance. It is surely 
reasonable that university teachers, including teachers of 
science, are stressing the importance that students should 
be able to write decent intelligible English before coming 
‘to the university. Apart, however, from the problem of 
standardising, there is a further difficulty in ensuring that 
students shall have the capacity. It seems possible for a 
pupil to write good English when he is thinking carefully 
about structure and style, and then to relapse when the 
thought of style and correct composition is ousted by the 
practical purpose of the moment. How else can one 
account for such facts as the following ? A student who 
had matriculated with credit in English literature wrote to 
a ptingipal of a training college as follows : 


“ DEAR S$18,— 
“Tam so very grateful to you in being so kind and for 
causing you so much trouble, and I cannot express my thanks 


for your great kindness for returning my acceptance form. 
“Тат, etc.” ? 


1 Op. cit, р. 47. The correlations given by Dr. Crofts are about 0-75 for French, 
Latin, and mathematics, and about o:65 for English, history, and geography. In 
one examination the average correlation for three years, of English composition 
in School Certificate with the schools’ estimates, was only about 0'4. 

2 This is taken from a “ Discussion on English in the School Certificate Examina- 
tion as a qualification for admission to Training Colleges,” by S. Hoole, Forum of 
Education, vol. v, 1927, p. 46. 
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The unteliability of the marking of the essay is attached 
to some extent to all subjects in which answers involve 
English composition as an important factor. "Thus the 
report of an enquiry as to the reliability of examinations in 
English literature, which reaches me as I revise the draft of 
this chapter, shows that seven experienced examiners mark- 
ing fifty-seven scripts in English written in the Durham 
University School Certificate examination gave most vaty- 
ing judgments on many candidates. Thus seven candidates 
who were failed by two examiners were awarded among 
them by the other examiners twenty-two passes, sixteen 
credits, and one special credit. 

The instability of the marking of examination papets has 
been thrown into relief by the remarkable constancy of the 
marking of mental tests. Now it seems to me that mental 
tests can never replace examinations. For we need, as a 
rule, tests of attainment as well as tests of capacity, especi- 
ally for entry upon university courses. At the same time 
it is remarkable how reliable mental tests have in some cases 
proved, even at the university stage; for example, at 
Columbia University the performance of the students in 
the intelligence tests given on entry proved a more 
reliable prophecy of their success at the end of their 
first year’s work than did the results of the entrance 
examination, and much more reliable than the secondary 
school marks. As to the authorities at Columbia it was 
wtitten : 

“From an attitude of healthy, if not severe, scepticism 
towatds the use of intelligence tests for this purpose, the 
whole college administration came, within the space of two 
years, to consider the intelligence tests as an indispensable 


® Full particulars of this enquiry are given in the Journal of the Assistant Masters’ 
Association, December 1931. 
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part, not only of the admission machinery, but also of the 
administration of the college in the Dean's Office.” 1 

I mention this example, not as an argument for mental 
tests for university entrance, but to indicate one good 
quality in mental tests, namely, their freedom from subjec- 
tive variations in marking, and the possibility of exact com- 
parisons. If tests which have so little relation in context to 
the knowledge required for university studies are so reliable 
a guide to success, in spite of the fact that they do not test 
interests or qualities of character, e.g. conscientiousness and 
persistence, as examinations do, then the tests must be very 
accurate from the point of view of marking and standardisa- 
tion. And, indeed, so it is found. In a large city, where 
the entrance examination to secondary schools is supple- 
mented by mental tests, the marking of the latter is done by 
clerks, while of course the marking of the papers has to be 
done by specially selected examiners, with a skilled chief 
examiner to standardise results. 

Yet the unskilled clerks produced in far less time, from 
the results of the mental test, an order of merit which was 
thought to be so like that given by the examination that it 
was suggested by some that the mental tests were unneces- 
saty, and could be dropped. Wisely the authorities 
decided to tetain them, as a check on futute examinations. 
It might, indeed, just as well have been suggested that the 
trouble and expense of the elaborate examination could be 
dispensed with. But we have not sufficient evidence as yet 
that tests are adequate for such selection, and even if we had, 
public opinion ïs by no means ready for the dropping of the 
examination test. Many parents and even teachers resent 


1 See R. Pintner, Intelligence Testing (London, 1924), p. 276. At the end of 
the chapter are numerous references to papers recording the results of the applica- 
tion of tests to college students. 
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the use of such novel tests for which the children cannot 
be prepared. 

Later we shall consider briefly the evidence so far avail- 
able as to the reliability of mental tests for the selection of 
pupils for secondary schools, or as a supplement to the 
ordinary entrance examination in English and arithmetic. 

A fourth reason for suspecting the reliability of some 
examinations as a comparative estimate of the ability of 
candidates is that pupils come from different schools, in 
which the standard of teaching (in different schools) may vary 
greatly ; so that children who have been specially well taught 
(or at least well taught from the point of view of preparation 
for the examination) have an advantage. ‘This criticism 
applies more particularly to the admission examinations 
fot secondaty schools. 

Incidentally this fact has an unfortunate repercussion on 
some schools in areas where the entrance examination is 
regarded as a test of the work of the elementary schools. 
This is unfair to those schools who draw their pupils from 
poorer homes, where not only are the conditions for work 
less favourable both physically and mentally, but where the 
children may possibly be of lower inborn mental capacity. 

A fifth ground for suspicion as to the reliability of selec- 
tion, both of pupils at тт + for the secondary schools, and 
to a less extent of scholars for the universities, is special 
cramming. 1 find a universal conviction among teachers 
whom I have consulted that some parents have their chil- 


1 І may record, as an example, the criticism of a parent about the following test : 
“ A guard in an express train walks from the back van to the front, in the direction 
in which the train is going. Then he turns and walks back, this time walking 
against the motion of the train. On which journey does the guard walk the 
farther?” An indignant parent actually wrote to the Press about this, and said 
that he had put the question to a man who had been a guard for twenty years, 
and even Ze could not answer it. 
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dren of 10 ot 11 specially coached out of school hours for 
the entrance examination to secondary schools, in the hope 
especially of getting free places and maintenance grants. 
As I write I have before me a book just published, the 
express purpose of which is to help in this special prepara- 
tion for entrance scholarship examinations for secondary 
schools: it consists of test papers, many of the questions 
being.culled from examination papers set by various local 
authorities in such examinations. Such ad hoc preparation 
within the elementary schools or outside is mentioned by 
some heads of secondary schools as one of the reasons why 
some pupils near the top of the list in the entrance examina- 
tion fall behind (in the secondary school) other pupils 
who have not been specially crammed for the entrance 
examination.! 

At the universities the point is put somewhat differently. 
It is that certain students gain admission and others gain 
scholarships, because of the special thoroughness of the 
preparation given in their schools; but they are ‘ spoon- 
fed,’ and often unable to adapt themselves to the freer and 
fuller methods of study at the universities.? 

A further point of criticism in reference to the entrance 
‘examination at 11 -+ for secondary schools is that the usual 
Simple examination in English and arithmetic is incapable of 
Selecting aright the pupils who are most fitted for the varied 
secondary school curriculum—who possess in the right degtee, 
not only sufficient genera/-intelligence, but also the specific 


1 Ina class of 74 graduate students I found that 34 had direct personal knowledge 
of the special coaching—out of school—of elementary school pupils preparing for 
an entrance examination into a secondary school. This proportion is especially 
significant, in that a substantial part of the class had never been pupils in an 
elementary school. 

2 Cf. the recent statement by a vice-chancellor of a modern university (Sir 
Charles Grant Robertson) to the effect that our university students come up to the 
university over taught. 
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abilities needed to do good work in foreign languages, 
mathematics, and science. 

Most diverse views exist among teachets as to this. 
Some heads of secondary schools (boys’ and girls") with 
whom I have discussed this ate confident that the requisite 
abilities cannot be selected at 11 + on the usual examina- 
tions. Some others seem equally confident the other way. 
The question will receive further consideration in later 
chaptets. 

Lastly, there is the general criticism that, though exami- 
nation success implies certain types of ability and certain 
valuable qualities of character, there are other kinds of ability, 
required and revealed not only in later life but in the life of the 
school, which examinations do not test. "The kind of criticism 
I have in mind is often expressed by business men who 
speak of ‘ practical ability’ or ‘common sense,’ as con- 
trasted with academic ability; but it is also expressed 
strongly by some experienced teachers, of whom I may quote 
as an example the late Headmaster of Bromsgrove School, 
Mt. R. С. Routh, who on the occasion of his last prize- 
giving said, as to the School Certificate, that “ not only did 
it keep a number of people out of posts for which they were 
very well fitted, since it was regarded as essential fox entry 
into the Services and professions, but it was a test only of 
certain academic achievements which could be passed with 
great ease by some boys at the age of fifteen, but which 
others, by no means their inferiors in many respects, found 
the greatest difficulty in passing at the age Qf eighteen." 1 

The problem is worthy of more systematic enquiry. On 
the whole, few will doubt the existence of a general relation 
between academic.success (as compared with bad academic 
failures) and success in the affairs of after life. One enquiry, 

1 Reported in the Birmingham Post, June 27th, 1931. 
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made by Dr. Edgar Schuster, into subsequent success in 
the professions of the Law, of the Church, and of Teaching, 
showed a decided superiority in professional status of those 
who had obtained first-class honours at the universities as 
compared with those who had only obtained third-class 
honours.? 

On the other hand, the recent confessions of Mr. 
Winston Churchill and Sir Eric Geddes, though they may 
be somewhat embellished, recall to mind other men 
who were for one reason or another failures at school, 
but who revealed marked ability later. 

This question, however, is wider than our own immediate 
problem, which is to test the reliability of examination 
success at one stage by examination success at a later and 
presumably mote reliable stage, more reliable in the sense 
that it is a truer estimate of the man as he will become. 

Under this head a reference should be made to one aspect 
of thé problem seldom thought of. Are there not some 
excellent qualities which actually lead to poorer perform- 
ance in examinations? Occasionally I have learned how 
the unselfishness of students in devoting time and energy 
to helping at home, especially during times of sickness or 
special poverty, has necessitated the neglect of university 
work; and the same thing no doubt occurs more fre- 
quently among senior school pupils. Also there is to be 


1 See Eygenics Laboratory Memoirs, 1-5 (London, 1907). Dr. Schuster blended 
his figures for Teaching and the Church, and found that ‘ distinguished positions * 
(Bishoprics, Deaneries, Professorships, Headships of Public and First-class Grammar 
Schools, etc.) were obtained by 68 per cent. of the first-class honours men within 
the whole group, but by only 32 per cent. of third-class honours men. One would, 
however, in any case expect a close relation between high honours and such posts 
in the seaching profession. In the Law the figures were less striking, 46 per cent. of 
the first-class graduates gaining positions of distinction, against 22 per cent. of the 
third-class honours men. In all, 3,508 men were traced under ‘ Church and 
Teaching,’ and 634 under ‘ Law.’ 
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reckoned the service given to school or university in 
various official positions. Once, after I had lectured to a 
University Students’ Society on some of these aspects of 
examinations, one of my own students came up to ask my 
advice. He told me that his university teachers gave him 
to understand that if he worked his utmost he was fairly 
sute to gain first-class honours. He was, however, giving 
a good deal of time to university official matters (including, 
I think, some post connected with the University Christian — 
Union), and he was quite certain that if he continued this 
work he would be unable to gain the coveted ‘ First.’ 
What, he asked, was he to do ? 

No doubt this aspect of the problem is not important 
for most students It merges, however, into one of 
more general influence, the examination teward gained, 
especially at earlier (school) stages, by narrow concentra- 
tion on examination work, to the neglect of wide human 
interests in the affairs of everyday life, in politics and social 
problems, or in the Church ; in scientific and other hobbies, 
ot in wide general reading. Such interests may tell 
favourably at later stages in life, and indeed they begin to 
tell in some subjects at university stage, in a. way they 
do not in the more elementary work at school. 


SOME PRESENT-DAY PROPOSALS 


Distrust of the reliability of examinations and the wish to 
avoid some of their harmful influences, and especially to 
avoid the hard-and-fast distinction between pass and fail in 
the School Certificate, have led recently to some striking 
proposals. For example, Dr. Edwards suggests that to 


i The Vice-Chancellor of Birmingham University says that he regards this 
as а very important problem for many of the very best students. ; 
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evety pupil leaving school at the age of 16, there should be 
given a School Certificate showing a recotd (a) of his work 
and success at school, and (4) of his performance in the 
public examination ; but without classification into * passes ? 
ot ‘ failures? 1 Again, а very important committee of the 
New Education Fellowship goes so far as to say that the 
right line of development is “towards the award of the 
school-leaving certificate by the schools themselves—such as 
ate certified as worthy to be trusted with this task.” 2 

Even at university entrance stage there are suggestions 
made that the present tests are unreliable, in that they let 
through some who are metely crammed and ‘ spoon-fed,’ 
but who ate lacking in general intelligence and especially 
in initiative and independence of thought. Consequently 
some ate proposing that school reports or some estimate of 
personality should have weight in deciding admission to, 
and scholarships for, the universities. 

Such revolutionary proposals now © in the air’ make all 
the more important an attempt to discover as fat as possible 
the facts concerning: the reliability of examinations. 

Finally, the question of the reliability of examination 
awards both at 11 + and to scholarships to the universities 
is of püblic importance, because of the large sums of public 
money now spent on free places and maintenance grants at 
secondaty schools and on university scholarships. 

From the evidence to be given later it certainly looks as 
though a considerable portion of such public moneys is 
being expended on the wrong persons or for a wrong 


purpose. 


1 See his pamphlet, 4 New Proposal, in reference to the School Certificate. 

? Interim Report of Examinations Enquiry Committee, 1931. 

з This is already the method followed in admitting to some (and I imagine all) 
of the Education departments of universities. 
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It is with a view to supplying some evidence on these 
questions that I have given in this book an account of an 
enquiry into the reliability of each of these levels of selection 
by examination. 


Тнк PRESENT ENQUIRY 


First we shall enquire into the reliability of the public 
examinations for pupils at about 1x yeats of age who seck 
admission (and free places or scholarships) to secondary 
schools. We shall judge this reliability by the success of 
those pupils in the secondary schools themselves, so fat as 
that success can be judged by the position in school at the 
end of the fourth or fifth years, by success ot failure in the 
School-leaving Certificate examination, and also by refet- 
ence to the reports of the heads of the secondary schools. 
We shall ask—' Do the pupils who do best in the entrance 
examination at 11, also do best in the examinations ot the 
school work at 16 years of age?" — | 

'The reliability of this School Certificate examination will 
itself be tested by the estimate of pupils by the schools 
themselves. Later, the reliability of the examination taken 
tight at the end of the school petiod, i.e. the Higher School 
Certificate examination and other examinations for scholat- 
ships at the universities, will be tested by reference to the 
results gained at the end of the university period. We shall 
ask— Do those who secure Scholarships at about 18 
maintain their pre-eminence and justify their awards in 
the final university examination at 21 ої 22; do those who 
secure scholarships on entering the university secure 
better degrees than those who do not ? ? 

It may be argued that any given School Certificate or 
even degree examination is an incomplete means of estimat- 
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ing the success of a pupil at that stage, and therefore is an 
incomplete test of the reliability of the entrance ot scholar- 
ship examination on which the pupil’s selection is based. 
This is no doubt true ; but in itself it is a striking admission 
of the fallibility of examinations. 

We must bear in mind, moreover, that final dégree exami- 
nations ate searching tests involving usually over half a 
dozen papers in the final year (in addition to earlier tests) : 
they are taken when students are relatively mature; and 
some elements of luck ate reduced by their being internal 
examinations. As regards the School Certificate, the result 
is based usually upon six or seven subjects instead of merely 
upon English and arithmetic as in most entrance examina- 
tions for secondary schools. 

Furthermore, for our most important enquities we shall 
teduce the clement of luck and the variability of marking, 
etc., by combining the results of a given School Certificate 
examination with the school’s own order for the same 
pupils, so that each pupil is examined in each subject twice, 
a dozen or fourteen papers in all. 

It should be made clear at once that this book is no 
violent appeal for the drastic reduction of examinations. 
We do not ignote their value. It may be admitted that 
some fottns of public examination for the above purposes 

fe essential, and will probably always remain essential. 
*In view, however, of the enormous influence of these 

lective examinations on the cateets of pupils, it is im- 
ottant that we (including the general public) should have 
as clear an understanding as possible as to what these 
examinations mean and as to their reliability. The losing 
of a School Cettificate, dependent as it may be on the loss of 
a few marks which might have been given by another 
examiner (ot gained by the setting of a different question) 
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may mean prevention ftom entering a chosen profession. 
The award of a scholarship at a university to a student who 
cannot afford to go without one, may mean an entirely 
different future career. At least as critical is the selection 
at 11 years of age of a pupil in an elementary school. 
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studied, judging from several lines of evidence to be given 
later. 

Let us take another and quite typical centre. Here the 
figures were still more striking. There were, in the 
entrance examination of this centre: (т) intelligence tests, : 
(2) written examination papers in English and arithmetic, 
(3) an oral test. A maximum of 100 marks was given for 
each of these three parts. If we teduce this total maximum 
of 300 to 100 we get the following figures : 


Top boy, (1) 78 per cent. (12) | 
Second, (2) 72 percent. i) 61 pet dent. 
etc. (3) 70 per cent. (14) 
(4) (15) 
e 67 per cent (19) : 
(6 66 per cent. (17) aper сео: 
(7) (18) 
(8) (19) $9 pe: cent. 
(9) > 65 per cent. (29) 
(хо) (21) + 58 ре cent. 
(11) eal 


(23) 57 per cent. 


The first 18 boys (with бо pet cent. of the marks of over) 
are the boys who obtained free places. Notice how close 
they ate; especially after the first five. After number six, 

-12 bews ate only separated by 3 рег e at the most! 
Number nineteen, only 1 per cent. behind, has to pay fees, 
if he enters at all. We may be sure that, with another 
examination the next week, ог even a té-marking of the 
same papers by different, examiners, the just-fails might 
easily replace the just-passes. I may add that one of the 
four boys who only just scraped in with 6o per cent. proved 
the best of the lot five years later, bein $ top of the school 
order, top in the School Certificate exa f ination order, the 
only one of the year to gain matriculati h, and the only one 
a prefect | 








46 SCHOOL ENTRANCE EXAMINATIONS [Chap. ш. 


Let us now turn to a question of practical importance 
intimately connected with this closeness of the pupils 
around the admission-level mark. 


THE SUBSEQUENT PERFORMANCE OF Boys WHO ONLY JUST 
GAIN ADMISSION 


What is the subsequent performance of the very f tail * of 
the admissions into the secondary school? H the perform- 
ance of the tail is good, then we cannot be sure (in view of 
the facts we have just reviewed and which will be greatly 
amplified later) that we are not excluding many pupils from 
the secondary school who are quite fit for it. The perform- 
ance in the School Certificate of boys at the top, in the 
middle, or at the bottom of the entrance examination will 
be considered in detail, and for eight centres, later. Here, as 
a preliminary example, I will give the performance of the 
tail (the last three admitted) in the two centres from which 
the above figures were taken. 

In the first mentioned, I had reports from seven schools. 
Three, however, only give results of School Certificate four 
years after entry, and it does not seem fair to say that pupils 
of the age of тї + at entry prove unsuitable for secondary - 
school education if they have not obtained a School 
Certificate by the age of 15 +. Many: boys in the public 
schools do not teach the standard of School Certificate 
before 16 ог even 17 years of age. We will confine out- 
selves, therefore, to tesults gained after five years in school. 
(These I have obtained for twelve examination-yeats from 
four schools with 1 rge entry lists, varying from 63 to 99. ) 

Taking the 36 p pils who constitute the bottom three in 
the twelve various ntrance examination lists, we find that 
17. gained a School "ertificate in either four or five years 
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(one of them reaching matriculation standatd). In view of 
the closeness to one another of the pupils at the bottom of 
their entrance lists (separated as they were by only 1 per 
cent. of the marks), it is fairly certain that some pupils just 
below, and possibly well below, the entrance level, would 
have also gained a certificate. This becomes still more 
probable when we find, as we shall later, that there is so 
little relation between the order of merit in the entrance 
examination and the ordet of merit in the School Certificate. 

Even this centre then, where there was large accommoda- 
tion for secondary pupils, where the entrance examination 
was most efficiently conducted, and was one of the most 
successful (judging by more exact evidence to be given 
later) in discriminating between the pupils at 11 +, there 
seems to be a decided probability that many pupils not 
admitted would have justified their admission—if we take 
obtaining a School Certificate in four or five years to 
imply that. 

In the second centre for which the marks were given 
above, I found that of the bottom four boys bracketed 
equal in the list given, every one gained School Certificate, 
and one, as already mentioned, matriculated—though the 
top bey in the entrance examination failed even to get 
School Certificate ; while in another year, two of the three 
bottom boys gained School Certificate (one with matricula- 
tion). It seems clear then that in this centre also (and the 
school was the only secondary school for boys in the town) 

pupils just excluded from free places might have amply 
justified the award had it been given them—because for 
practical purposes we cannot regard ihe just-fails at 
entrance as appreciably worse than the jus -passes when the 
lists of marks are of the type given. 

These results are confirmed by indivic.ual reports, which 
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show that some borderline pupils, who only just gain 
admission, do exceedingly well, and sometimes fat better 
than most of the pupils much higher up in the entrance 
examination. One of these I have already quoted. The 
Head of another school, whose results have been given 
elsewhere, stated that one boy who was eighty-eighth on the 
list for his school, and barely escaped refusal, proved by far 
the most brilliant boy of the year, matriculating at 15 with. 
three distinctions. In another centre, of the 25 boys taking 
the School Certificate examination, the one who had been 
the twenty-fourth in the entrance examination was the only 
one who obtained first-class honours, and one of three to 
teach matriculation standard, Another Headmaster told 
me how a senior mistress had begged for the admission of 
the next pupil on the list, on the ground of personality and 
home background, though he felt they could take no mote. 
After continued pressute he agreed—and the girl proved 
one of the best they had had, on general grounds, for 
several years. 

We have exemplified this point so far by reference only 
to two examination centres. ‘The totals for the ten schools 
(in five centres) from which I got reports as to the bottom 
three pupils at entrance were as follows : 


\ 
Number of pupils id bottom three at entrance А А S tss . 67 
Number obtaining 5 оо] Certificate. . . à . А . 30 


'Taken together, these ате fairly representative of the ten 
centres from which statistics were obtained for this en- 
quiry ; indeed, s from the full evidence which will be 
given shortly, tháse particular centres had decidedly more 
telidble entrance xaminations than the average. Of these . 
centres, then, we зау say that, at this most critical stage of 
a pupil’s life, wher. the question of obtaining a secondary 
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education or not is for the great majority finally settled, we 
must suspect that a number of children are being excluded 
from admission to secondary schools (ot free places) who 
would profit from them as much as many admitted. 


ARE THE ENTRANCE EXAMINATIONS ADMITTING PUPILS 
UNSUITABLE FOR SECONDARY EDUCATON ? 


Before going on to our main enquiry, I may touch on 
this second problem of immediate practical importance. 
As would be gathered from Chapter I, I should not per- 
sonally regard the failure to obtain a School Certificate as 
proof of a boy’s being unsuited for secondary education ; 
those who do will find ample evidence later of the admis- 
sion of such unsuitable pupils. Indeed, it is already known 
that a large proportion of the pupils in the secondary 
schools leave without a School Certificate! 

There is another line of evidence, however, as to unsuit- 
ability. I obtained from the Heads of six large schools 
statements as to how many pupils left the school before 
four years with bad reports on their work: I was surprised 
to find that the numbers were 140 out of 797, i.e. about 
тт per, cent. (These do not include those who left for 
economic or other reasons, but whose work was reported 
as satisfactory—who formed an additional 9 per cent.) 
This means that over one-sixth of the pupils admitted to 
these schools were unsatisfactory. They can hardly be 
regarded as suitable pupils for secondary schools with their 
existing curricula. Again, these schoo]s were actually 
above the average from the point of view of reliability of 


1 See the Board of Education Report for 1930 : Section C, on Secondary Schools. 
Of those who reach the School Certificate class and sit for the examination, just 
over 70 per cent. obtain a certificate ; but many of coi 'rse leave before reaching 
that class. i 
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their admission examinations, judging by the evidence to 
be given latet. Finally, when we look back to the entrance 
examination records of these unsatisfactory pupils, we find 
they wete not massed near the bottom ; they were not those 
who just scraped in. They are found exactly as often in the 
middle group at the entrance examination as in the bottom 
group—and only slightly less often in the top group. 
The actual percentages for the 140 bad reports wete : 


Top third at entrance А А .. 14'3 per cent. had bad reports. 
Middle third at entrance . . . gr per cent. had bad reports. 
Bottom third at entrance . > - 192 per cent. had bad repotts. 


Thus in entrance examinations even better than the 
average of those tested, the examination order fails to dis- 
criminate clearly between the good and the bad, and fails 
to exclude many who are apparently unfit for secondary 
education, taking as our criterion the report of the Heads. 
Here again then we see how important it is that the order of 
metit at the entrance examination should be a reliable one. 

We will now proceed to consider the reliability of these 
various entrance examinations mote carefully, and in 
patticular to use more objective criteria than the reports of 
the Heads, valuable as these undoubtedly are. 


Chapter IV 


THE METHODS OF OUR ENQUIRY AND RESULTS 
FROM FIRST CENTRES 


METHODS OF TESTING THE RELIABILITY OF THE ENTRANCE 
EXAMINATION 


THERE are sevetal ways of testing the reliability of the 
entrance examination at 11 +, besides those used in 
Chapter III. 

(а) The first and crudest method is to find whether the 
pupils who did best in the entrance examination were more 
successful than the weaker ones in gaining a School Certifi- 
cate four or five years later. 

(P) Another way is to compare the order of merit of 
pupils in the entrance examination with their order of merit 
in the secondary school at 16 ot 17, based on the schools’ own 
examinations, combined perhaps with marks of the weekly 
work for the last term ot two. 

(с) A third method is to find the order of merit in the School 
Certificate examination, and compare that with the order of 
merit of the same pupils in the entrance examination. This 
57 course takes the School Certificate as a criterion of 
success, and few would accept that as final. But by con- 
sidering the sum total of marks gained in the School Certifi- 
cate examination (instead of merely whether the certificate 
was gained or not) one can.lessen the element of chance to 
some extent. Most would admit that the, performance in 
the School Certificate examination, if one takes sufficiently 
large numbers of pupils in various areas, id one useful index 
of success of the pupils ; at least, to deny this is to admit an 
unreliability of examinations greater thah is suggested in 
this book. 
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(d) A fourth and more accurate method of testing the 
prophetic value of the entrance examination is to blend the 
order of merit in the school certificate examination with the school’s 
own order of merit at that stage, based on the school’s own 
examination (or examination plus weekly work). This 
averaging of two examinations should certainly lessen 
etrots due to ‘luck,’ to varying standards of marking, and 
so forth. 


THE METHOD or ENQUIRY 


My first step was to write to a number of Heads of 
secondary schools, most of them known to me personally, 
to ask if they could supply the following information : 

(т) The order of merit of pupils in the entrance 
examination in a given year—say 1924. 

(2) The order of metit of the same pupils still at school 
in the School Certificate examination four or 
five years later; and whether the School 
Certificate was gained. 

(3) The school’s order of merit for the same pupils at 
School Certificate stage, based on the term’s 
work or school examinations, or both. 

These particulars were asked for three or four recent 
and successive years. 

As to calculating the order of merit in the School Certi- 
ficate examination, the following suggestion was made : 


INSTRUCTIONS FOR НЕАРМАЅТЕВЅ 
The examinatiqn marks may be given according to the 
scheme the Head thinks fairest in his own school. But the 
following method, suggested by one Head, is recommended : 
(a) Take the thtal marks of the best six subjects (or seven 
if all pos in school take seven). 
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(P) Ааа half the marks gained in any additional subject, 


even if the examination in that subject is not passed. 


In practice this question of the basis of the order proved 
a simple one, for the usual custom seems to be that all 
pupils in a class take the same subjects, though occasionally 
an extta subject is taken by a few pupils. In these latter 
cases it was felt that if an extra subject was attempted some 
ctedit should be given for it, even if the examination in it 
was not passed ; but that to add all the marks gained in it 
was to give too great an advantage to the pupil attempting 
an extra subject. : 

Results of this enquiry were sent me by 22 schools in 
different parts of the country, from Lancashire to Devon 
and from Kent to Monmouth. They dealt with entrance 
examinations conducted by ten different authorities, and 
comprised chiefly entrance examinations during the yeats 
1923 to 1927. Many other Heads would have helped in 
the enguiry, but had not particulars of the enttance orders 
of pupils sufficiently far back. Where Directors of Educa- 
tion possessed these particulars they were almost invariably 
most willing to co-operate. Sometimes, however, full 
mstignlars were. not available. 


1 This and several other points in the method of enquiry were decided on in 
consultation with four Headmasters, with whom I discussed fully this enquiry, and 
to whom I tend my warm thanks for the help given at this esrly stage—Mr. J. К. 
Brown, Headmaster of George Dixon Secondary School, Birmingham; Mr. Н.Р, 
Lunn, Headmaster of Yardley Secorfdary School, Birmingham; Mr. J. Manton, 
Headmaster of King Edward’s Grammar School, Aston ;; and Mr. H. Watson, 
Headmaster of Dudley Grammar School. Another impoftant point which they 
discussed with me was tb^ question of taking the order at ¢ntry as given bya Local 
Education Authority’s examination, which usually ташы an allowance fot age, 
It was felt that this was the right course to adopt, for rea fons stated in the note on 
Age Allowances in Appendix I (0), р. 182. It is there s own also that our results 
are unaffected, even if this age allowance is ignored. 
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THE GAINING OF A SCHOOL CERTIFICATE AS A TEST 


Let us consider the simplest way of testing the relation 
between success at entrance examination and success at 
School Certificate stage. 

Suppose we take the lists of pupils in order of merit at 
the entrance examination (ignoring for the time those who 
leave before sitting for the School Certificate examination), 
and divide the pupils into three groups according to their 
performance in the entrance examination. Thus: 


(а) The top group . А . : . One-third of the pupils. 
(2) The middle group 2 3 P . One-third of the pupils. 
(с) The bottom group  .. Ў E . One-third of the pupils. 


Now, if we consider how these groups fare in the School 
Certificate examination, we find that, in one centre, B for 
example, the figures for a range of four yeats of School 
Certificate examinations werte as follows: 


SCHOOL CERTIFICATE 





Passed 
In 4 years In 5 years 
T Е £5; 
Op groups е А . . | 25 out of 33 27 out of 53 
Middle groups . . . «| 25 out of 32 28 out of 32 
Bottom groups . . . ‚| 24 out of 33 31 out of 33 





We see that, though the top group is vety slightly better 
(by one pass) than the bottom group in the record at 4 
years, after 5 years the bottom group scores actually four 
mote passes {һай does the top group. Judged by this 
standard, the Боут group is at least as good as the top. 

Five out of the tight examination centres (namely A, B, 
C, E, and F) fot which such figures wete obtained showed 
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results of this kind. Adding their totals we get the 
following results : 


TOTALS FROM Five CENTRES 





S.C. gained in 4 years S.C. gained in 5 years t 








А Middle Bottom А Middle Bottom 
Top Thirds) Thirds | Thirds | TOP Thirds| Thirds | Thirds 


— 





50/99 48/92 42/99 74199 74/92 74199 





Percentages 50:5 | $22 424 748 805 748 





Неге we find the remarkable fact that in five of the eight 
centres thus tested, the boys in the middle group at the 
entrance examination beat those at the top at entrance, 
whether we considered the four- or the five-year courses ; 
while the bottom group do as well as the top group in the 
five-year results, and are very little behind them in the 
four-year results. 


THE ORDER oF MERIT IN THE SCHOOL CERTIFICATE 
EXAMINATION 


The actual gaining of the School Certificate is, however, 
an uncertain test; for as we have seen (in Chapter Т), a few 
matks more or less in a compulsory subject may make all 
the difference between passing and failing, and a pupil A, 
with a total of marks 5o per cent. greater than that of pupil 
B, may fail to gain a certificate, though B gets one. We 
can, howevet; test the reliability of the ehtrance examina- 
tion in a better way, namely, by comparing the order of merit 
of the pupils in the entrance examination| with the order of 

1 Two schools sent theit pupils in for the School Crtificate examination only 


after five years at school; in the other schools practically all students took the 
examination after four yeats, trying again the next year if they failed. 
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merit in the School Certificate, or in the schools’ own lists 
four or five years after entry, or in both combined. 

Now we can get a tough idea of the resemblance of two 
orders by comparing them directly as in Fig. П. There I 
have shown the order of metit of a class of boys of 11 + in 
the entrance examination in a certain school in Town A, 
and also the position of each boy at the end of five years. 
The arrow lines show whether the boy goes up or down as 


Top group at entry [| Middle group at entry | | Bottom group at entry 
Entry — Order бусаг || Entr, Order 5 s|] Ent d 
539 die || бла EP бй. OU aur 


тро = 
RU к= 


6 6 6 
{ 8 s 
9 9 9 
o o 


= 


Фое 





Fic. П. 


‘This order is an averape order got by blending the order of merit in the School 
Certificate examination with the otder in the school's own examinations held in 
the same term, ` 


he passes from the entrance examination to the examina- 
tions at the end of five years. 

As will be seen, we have divided the 15 bbys, as before, 
into three groups at the entrance examination—top, 
middle, and bottom. Now, if we inspect Fig. П, three 
striking facts emege : 

(1) Not one Dr the boys in the top group at entry 
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remains in the top group in the order of merit 
five years latet. 
(2) Only one of the boys in the bottom group at entry 
remains in the bottom group five years later. 
(3) Three out of five boys in the bottom group at 
entry ate, after five years, above a// the boys 
who wete in.the top group at entry. 


One can say by inspection here that there is little re- 
semblance between the two ordets—probably most people 
would surmise that there was a tendency for those who did 
well at entry to do badly five years later. 

It is not, however, always so easy to see the lack or 
ргеѕепсе of resemblance between two orders, especially 
when the number of boys in the class is large. Conse- 
quently mathematicians have devised a way of finding and 
stating the degree of resemblance or correlation. We need 
not here trouble with the method of finding the degree of 
resemblance, and fortunately the way of indicating it is 
quite simple. The degree of resemblance or * cortelation " 
between two orders may be indicated by a decimal figute or 
percentage. If the orders were exactly the same the figure 
would be r'o. A high degree of resemblance between two 
otders would be indicated by o7 ог o8. A very slight 
degree of resemblance only is indicated if the figure is 
below o:3 ог o. The figure representing the actual 
degree of resemblance of the two orders in Fig. II is 
— ото, indicating practically no cortelation between the 
two orders. 


1 The practical use of this way of reckoning the resemblance between two orders 
is so easy to understand, so frequently convenient, and so often met with nowadays 
in statistics about educational and psychological studies, tł at it is well worth while 
even for the non-mathematical to be familiar with it. B iefly we may say that if 
two orders, A and B, resemble one another completely “(the pupils being in the 
same order in B as they are in А), then their correlation is fr. If the orders are exact 
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It will give those unfamiliar with such correlation figures 
some further idea of their value if I mention that a group of 
half a dozen intelligence tests which can be applied in two 
houts to a class of boys will give an order of merit which 
correlates with the estimates of the intelligence of those 
boys made in collaboration by several teachers who know 
them well, to the extent of o-8 or more. Even the attempt 
to estimate the relative general intelligence of a group of 30 
boys (previously unknown) merely by means of a convetsa- 
tion of about ten minutes with each, gives a correlation of 
about o2 ot 0-25 with the estimates made by teachers who 
know them well: Specially trained interviewers may, on 
the basis of half an hour's interview, even produce an order 
of metit as to general intelligence of pupils which correlates 
to the extent of about o*5 with estimates made by persons. 
who know the pupils well.? 

With all the possibilities of error due to ‘ luck,’ to varying 
standards of marking of different examiners, and so on, the 
order of merit of pupils in the School Certificate examina- 
tion correlates with the order of merit in the schools’ own 
examinations (held the same term) to the extent of about 
o:8 on the average for all centres, as we shall see later. 


opposites, so that thle top boy in A is bottom in B, the second in A bottom but one 
in B and so on, the correlation is — 1. If there is no more resemblance between the 
two orders than would be given by mere chance, the correlation is o. 

The whole theory of correlations is thoroughly expounded by W. Brown and 
Godfrey Thomson in Essentials of Mental Measurement. An exposition which is 
especially useful on ‘the application of correlations to school examinations and 
masters’ estimates of '»upils is given in The Science of Marking, by Terry Thomas 
(London, 1930). a 

1 See, for example, E. H. Magson, “ How we Judge Intelligence ” (British 
, Journal of Psychology, МоўортарЬ Supplements No. 9). Also C. W. Valentine, “ The 
Relative Reliability of Ir uitive Judgments of Character among Men and Women,” 
British Journal of Psychology, vol. xix. (1929), p. 213. 

2 See Cyril Burt’s contribution to A Study in Vocational Guidance (Report No. 33 
of the Industrial Fatigue Research Board, 1926). 
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Furthermore, the order of merit after one year in the 
secondary school correlates with the order of merit in the 
School Certificate examination to the extent of about 0-69, 
as indicated later in Chapter VI. 

In view of these facts it is astonishing to find that in five 
of the eight centres in which the comparison was made, 
when the order of merit in the entrance examination was 
compated with the order of merit in the School Certificate 
examination four or five years later, the correlation on the 
average was practically zero. 'ТҺаї 15, there was no resem- 
blance between the two orders of merit, no tendency for the boys 
high in the entrance examination to be high in the School 
Certificate, or for those low in the entrance examination to 
be low in the School Certificate. The highest figute for 
any of these five centres (A, B, C, D, and E) was от). The 
lowest was actually a negative correlation, —o'15. The 
number of pupils in some centres was small, but the total 
number for these five centres was 235. The rough average 
correlation for these five centres was almost nothing, the 
actual figure being 0-03. (For details, see Appendix I (а), 
p.174) І may add that Centres A, B, C, and/E were four 
of the five centres in which the top groups atjentrance did 
slightly worse than the middle groups in secutig the School 
Certificate, and little or no better than the bottom groups. 
Centre D made no report as to ‘pass’ orf fail’ in the 
examination. | 

Tt was to give a more concrete idea as tq what this low 
cottelation means that I have shown alrefdy in Fig. П a 
graphic representation of the changes in order, which 
_ give tise to a correlation of about the same/mount (actually 
а slightly lower correlation, namely, — ofo). 
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RELATION BETWEEN ORDER IN ENTRANCE EXAMINATION 
AND THE SECONDARY SCHOOL’S OWN ORDER OF MERIT 
AFTER Four ов FivE YEARS 

Let us now take the other test of the pupils’ success in 
the secondary school, namely, the order of the same pupils in 
the school’s own examinations а few months before the School 
Certificate examination. In the school examinations the 
pupils’ papers are marked by their own masters; and to some 
extent the marking would be influenced, unconsciously, by 
the masters’ impressions of the pupils’ work through the year; 
so that the school examination to some extent may introduce 
a valuable addition to the mere examination test. Іа some 
schools, indeed, the school order was based partly on the 
work of the year; and in one a boy's order in the school 
examinations was slightly changed if it was felt that the 
examination order did not fairly represent his deserts. 

Of the five centres we have just considered, D gave no 
report of its own school examinations. The average corte- 
lation in the|fout other centres between orders at entrance 
and four or буе years later was o-08 for 221 pupils, again 
indicating practically no resemblance. (Details in Ap- 
pendix I (0), p. 175.) 

A mote accurate test, however, is obtained if we blend 
the order in the School Certificate examination with the 
school’s own ‘order for the same pupils at about the same 
time as the School Certificate examination. 

Blending th. orders for each of the centres just con- 
sidered, we об ма the following correlations. As this is 
the most teliabl figure, I give each centre separately. 


AVERAGE CORRELAT. )NS OF ENTRANCE ORDER AND “ BLENDED " ORDERS 


Centre A . . я . Я . — 0°06 (25 pupils). 
Centre В. as * > x є о'тт (80 pupils). 
Centre С. S " Е А Я 005 (66 pupils). 


Cente E. Y : Я А А 0'15 (36 pupils). 
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The figures still indicate no appreciable correlation, the 
average being 0:06. If we include Centre D (in which only 
the School Certificate order was obtained, and which gave 
an avetage cottelation of o-oo for 22 pupils with the 
entrance examination) we may say that in five of the 
centres the entrance examinations for the years concerned 
and in these representative schools, failed deplorably to 
indicate the telative future performance of the pupils 
admitted, so far as that performance can be judged by two 
very full and independent examinations, each in six of 
seven subjects, taken four (or five) years after entrance. 

We ate using as our criterion of success the School 
Certificate examination and the schools’ own examinations 
held about the same time. But how far can we trust 
these ? 

As a check on the reliability of these examinations, the 
School Certificate and the schools! own examinations at the 
_end of the four-year period, I found the correlation between 
the respective orders of merit in these two examinations 
themselves. The rough average correlations for the four 
centres just mentioned were: Centre A, 0:75 ; Centre B, 
069; Centre C, 0:87 ; and Centre E, o:87—an average of 
nearly о:8, indicating a high degree of resemblance between 
the two orders. The blended orders may; therefore be 
taken as tepresenting with a high degree of ‘reliability the 
examination capacity of the pupils in the subjects taken in 
these examinations. 

The entrance examinations then failed] badly in the 
prophecy of examination capacity four orffive years later 
in these schools. 

It is important to bear in mind that ѕос[/а result involves 
not the slightest reflection upon the schools concerned. 
The efficiency of the schools is practically irrelevant to this 
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particular point. And if it were not, I have good teason to 
know that they ate schools of high efficiency, several with 
a Headmaster (or Headmistress) who has distinguished 
himself among his fellows. 

The centres dealt with so far, though of course small in 
number, ate fairly representative of all types. They in- 
clude one fairly large city, two small towns, and two 
smaller towns under county authorities. It is true that 
only one school for each centre was dealt with; but in 
three cases it was the only‘ boys’ school in the town; in a 
fourth it was one of two; in the fifth one of three, and 
itself a co-educational school. Also, in some schools the 
number of pupils in a class was very small, but in every 
case the results of two, three, or four years (or classes) were 
studied; so that the average was based upon twenty-six 
correlations i in alli 


RELATION BETWEEN ORDER AT ENTRANCE AND THE 
PASSING OF THE MATRICULATION EXAMINATION 


Before considering other centres, a word must be said 
about one other comparison that may be made—namely, 
that between entrance examination and the securing of 
“ matriculation ” after four ог five years. 

As soon as we confine ourselves to matriculation 
successes, the numbers become so small for most of the 
centres that nes are not of much value. They will be 
considered móre carefully in reference to Centre J, where 
the numbers aie larger. 

Of the centre. A, B, C, E, and F, only 17 per cent. of the 
pupils sitting fo. the School Certificate examination in four 
years gained ma|riculation, and only 29 per cent. in five 


1 For the influence o£ selection on correlation, see end of Appendix I (a), p. 174, 
and especially AppendixiI (7), p. 184. 
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yeats, against 76 per cent. of the pupils in these same 
centres who gained the School Certificate. (Centre D 
made no return as to matriculation.) Evidently a test 
which results in the success of only a very small proportion 
of pupils is of little value as a criterion of the entrance 
examination as a whole, and especially of its reliability near 
the bordet-lines of pass and fail. 

If it be suggested that the severer standard for matricula- 
tion makes a good test for showing the very pick of the 
pupils, the results do not testify much to the reliability of 
the entrance examination concerned even from this point of 
view. In one centre (A) the bottom group at entrance 
gained more matriculation certificates than the other two ; 
in another centre (C) the middle group obtained the most ; 
in yet another (E) the bottom group practically equalled the 
middle. ‘Taken together as the table below shows, the 
middle groups did neatly as well as the top groups; while 
the bottom groups score 19 per cent. of successes against 
38 per cent. by the top groups. 

The figures for the five centres were as follows (Centre D 
made no returns as to matriculation) : 





Matriculation gained in 4 or 5 years by 








Centre A 
Centre B 
Centre C 
Centre E 
Centre F 


Total. 


Percentages 
LÀ LLL oe a 





; 
Looking at it in a different way we may bay that 19 of the 
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boys from the bottom groups at entrance beat 61 boys from 
the top gtoups (who failed to gain matriculation), while 
one-third of the middle groups beat over two-fifths of the 
top groups. Hence, even taking matriculation as a reliable 
general criterion, which the small percentages of success do 
not allow us to do, we see that the entrance examinations 
in these centres do not indicate reliably even the very 
ablest boys. | 

(The value of matriculation as a test is further discussed 


in Appendix I (d), p. 176.) 


THe NATURE OF THE ENTRANCE EXAMINATIONS IN CENTRES 
А, B, C, D, anD E 

Was there anything peculiar about the entrance examina- 
tions of the centres we are considering? One point im- 
portant for the efficiency of an entrance examination is the 
granting of an age allowance. It is obviously unfair to 
compare a boy of тоў with one of 12 without giving the 
former some allowance for age, and an age allowance is now 
fairly common in entrance examinations. Considering 
these five centres, we find that A, B, C, and E have age 
allowances, while D gave no age allowance. Evidently, 
the failure tb give a correlation cannot be attributed to 
absence of age allowance? 

We seem compelled to conclude that for some reason or 
other in these five centres of those eight for which cortela- 
tions were ob ained, the entrance examination, in the years 
studied, gave fyractically no guidance as to how the various 

2 As to age allowa ces, and the methods of making them, see references in the 
note on age allowance in Appendix I (), p. 182. 
2 In Centre E there was also an interview ; and in the second of the two years 


taken for Centre A the entrance examination included intelligence tests and an 
oral test, 
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pupils would do in their school career. What these reasons 
ate we shall discuss later. We shall consider in particular 
whether it is due to the factor of luck, or to instability of 
matking; or whether it is due to the varying rates of 
development of different boys—one going ahead quickly— 
one slowing off (and I shall bring evidence to show 
that this cannot account for the absence of correlation); 
or whether it is due to the difficulty of testing in a 
simple examination in English and arithmetic all those 
specific abilities involved in the study of the many subjects 
of the secondary school curriculum—especially foreign 
languages, mathematics, and science. 

So far as regards these centres, we cannot rely upon the 
selection of pupils, either for the award of scholarships 
or free places to the best pupils, or for mere admission. 
Those just on the border prove as good as those at the 
top. And in view of this we must conclude also that those 
just below admission may be as good as most admitted (or 
given free places), and better than some. 


Chapter V 
RESULTS FROM FURTHER CENTRES, AND SOME CONCLUSIONS 


Lzr us now tutn to the tesults gained in the other three 
centres in which orders of merit were also given both for 
entrance and School Certificate. These Centres I have 
labelled H, I, and J. 


CENTRES Н AND `I 


In Centre H, though there was no allowance for age in 
the entrance examination, there was an appreciable cortela- 
tion between the order at entrance and at School Certificate 
—namely, o:39 fot boys and o-60 for girls for the year of 
entry, 1923. The numbers of pupils were 24 boys and 15 
girls, so that the correlation figure for the girls is very un- 
certain! We may, however, agree that here is some 
evidence of a definite resemblance between the orders in 
Centre H.? 

If we take the alternate method of checking the entrance 
examination, namely, the performance at School Certificate 
of the top third in the entrance examination as compared 
with the middle and bottom groups, we find that each 
group scores nearly the maximum possible number of 
passes, the figures being for the respective groups 12/13, 
12/13, and 11/13. As regards qualifying for matriculation, 
the middle group does as well as the top group, but there is 
a falling off in the bottom group, the numbers being 7/13, 
7/13, and 2/13 tespectively. 

1 With small numbers of pupils mere chance correlations may occur, i.e. the 
‘probable error’ is igh. Thus positive correlations based on very small numbers 
are untrustworthy ; but the absence of a correlation with small numbers is suggestive. 
(See Note on Probable Errors, Appendix І (0), р. 175.) 


2 In Centre HI did not obtain the school’s own order, so that I cannot give the 
correlation of entrance: with blended final orders for that centre. 
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In Centre I the correlations in a girls’ high school were 
obtained for three years. The correlations of entrance 
otder with School Certificate order were as follows : 


Entry 1921, 11 pupils . . . . Correlation = 0-44 
Entry 1922, 13 pupils , . А . Correlation = 0-64 
Entry 1923, 16 pupils . А . . Correlation = 0°25 


a rough average of 0-44. 

Here the numbers of pupils in the various yeats were so 
small that these correlations, taking each year alone, are 
very unreliable ; and, again, the correlation is lowest where 
the number of pupils (year 1923) is highest, and so most 
reliable. 

Still, the average based on до pupils indicates a definite 
though still small resemblance between order at entrance 
and in School Certificate. Furthermore, the correlations 
of enttance order with the schools’ own order five years 
later were, for the same three years, 0-27, 0°55, and 0-34, an 
average of 0-39, so that the rough average of all six correla- 
tions is o:41. Also, the average of the correlation of 
entrance with the blended orders was 042. 

As tegards the mete gaining of the School Certificate in 
Centre I, the top and middle groups at entrance do about 
the same, the bottom groups doing less well, the numbers 
being respectively—top third, 14/ 16; middle third, 12/15 ; 
bottom third, 9/16. Also, the figures for “ matriculation ’ 
wete 8 for the top third, 6 for the middle, and 3 for the 
bottom group. 

Here is a second centre then in which ge correlation 
figures (on an average of three successive examination 
years) show an appreciable resemblance between the ozder 
at entrance and the order at School Certificate stage, and 
where the performances of the top thirds also show a 
decided supetiority to those of the bottom thirds in the 
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mete securing of the School Certificate (or matriculation). 
We have seen, however, that this last is not a very reliable 
test; and, further, 9 of the bottom group of pupils get the 
Certificate, while 2 of the top group fail to do so. ' 

The examination in Centre I, I may add, included a 
viva-voce examination. | 


CENTRE ] 


I now come to Centre J, in which I secured the largest 
number of results based upon twelve schools. Let us con- 
sider first the performance in School Certificate of the top, 
middle, and lowest groups of pupils at entrance. The 
totals here given ate based upon the figures for two, three, 
or four different years in the various schools, and the figures 
relate to the School Certificate examinations of 1927 to 1931 
inclusive (i.e. entrance examinations of 1923-1927). 


SCHOOL CERTIFICATES GAINED IN CENTRE J 
In 4 years after entry, In 4 or 5 years after entry. 


Total of тї schools. ‘Total of xo schoolst 
Top thirds . Р Я 142/284 р 215/284 
Middle thirds . В . 102/286 179/286 
Bottom thirds, . А . 63/284 150/284 


Expressed as percentages, these figutes are as follows : 


Percentage gaining S.C. 


In 4 years. In 4 or 5 years. 
Top thirds — . : : 50°0 157 
Middle thirds А . Š 356 62:6 
Bottom thirds > А 22:2 52:8 


y 
On this rough grouping we get a decided superiority of 
the top groups to the bottom groups, and a superiority of 


1 One school gavel only results of four-year courses, and so does not contribute 
to the five-year results. It will be noticed that the totals of the middle thirds are 
greater than those of|the others. In dividing the classes into thirds, there was 
often, of course, an oc d boy or two, and these were allocated to the middle group 
more frequently than d the others, 
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the middle to the bottom groups, a better result than that 
of the other centres. 

Yet here again we find that over one-fifth of the pupils 
in the bottom groups at entrance beat one-half of those in 
the top groups in gaining the School Certificate after four 
years at school; while after five years over half the bottom 
groups have beaten one-quarter of the top groups. Further- 
mote, it has to be borne in mind that pupils low down at 
the entrance examination may sometimes be placed in a 
five years’ course fot the School Certificate, while others 
above them at entry, even if not teally abler, were placed 
in a four years’ course, so that the former would have no 
chance of gaining School Certificate in four years. 

Instead of School Certificate awards, let us consider the 
gaining of matriculation standard after five years from 
entrance, which gives us numbers comparable to School 
Certificate gained after four years. We find the per- 
centages ate as follows : 


PERCENTAGE GAINING MATRICULATION IN Four Оң Five YEARS 


Top thirds š í : . 121/284 = 42:6 per cent. 
Middle thirds . . . 81/286 = 285 ,, 4, 
Bottom thirds . А . . 42/284 = 148 , у 


Again a decided superiority of the top group, though 
one-seventh of the bottom groups beat half the pupils in 
the top groups.? 


ORDER AT ENTRANCE AND Four ов FIVE yis LATER 
IN CENTRE J 

Let us turn to consider the actual со Найз between 

the entrance examination order and the orderi in the School 


1 The use of matriculation as a criterion of success is a pmplex though interest- 
ing problem. Some further figures and discussion are d ven in Appendix I (2), 
р. 176. 
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Certificate examination in Centre J. (The details of these 
ate given in Appendix I (е), p. 179.) We will take only the 
classes in which the number of pupils taking the School 
Certificate examination was 18 or over. This restriction 
still left me with twenty-six classes from twelve diffetent 
schools, the tọtal number of pupils being 625. 

The average of the correlations between these orders at 
entrance and the School Certificate was о:4о. This figure 
is decidedly better than that of most of the centtes ; slightly 
lower than for H and I, but more reliable because based 
upon matty more classes and pupils. The correlation indi- 
cates a definite tendency for those who were high in the 
entrance examination to be higher at School Certificate 
stage than were those that were lower at entrance; but 
the resemblance of the two orders is still small. 

Let us consider the supplementary test—the order of 
merit of pupils in-the schools’ own tests, four or five years 
after entrance. This school order was in two schools 
based half on the records of the year’s work and half on the 
schools’ own examinations at the end of the School Certifi- 
cate year; in another it was based primarily on the exami- 
nations, but the order was adjusted somewhat if it was felt 
by the teacher that a pupil had markedly failed to do himself 
justice; in the rest of the schools concerned the school 
order was entirely based on the schools’ examinations. I 
obtained school ordets of this type from nine schools in 
Centre J. Te average correlation of fourteen such school 
orders (with cider at enttance) from these schools was 0:39, 
agreeing very closely with the average correlation (0-40) of 

‘the School Certificate orders of the same pupils with 
entrance orders) (Боғ details, see Appendix I (f), p. 180.) 
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55 
RELATION BETWEEN ORDER AT ENTRANCE AND “ BLE NDED 
ORDER AT SCHOOL CERTIFICATE STAGE 


As a further check, however, let us blend the orde £ in the 
School Certificate examination with the school’s ow 2 order 
—in the hope (as with earlier centres) that one tes t will to 
some extent counteract the chance errors of the ot} It. Of 
fourteen blended orders of this kind which I obt ained in 
Centre J, the average correlation with the entrance, examina- 
tion was 0-39 (again in classes of 18 or over): These 
fourteen classes, however, were not quite representative 
of the whole group: the average of their correlations of 
entrance otder with the School Certificate alone Was only 
0°36 against o-40 for all the classes taken in J. (Details in 
Appendix I (f), p. 180.) They did not includ the д 
yeats for some schools, when the correlations increased. 

The fact that the blended orders gave approximately the 
same correlation with entrance examinations, 45 did the 
School Certificate orders and school orders fot this same 
group of schools and classes, confirms the steadiness of 
an average of a large group of correlations jfor School 
Certificate. | 

Seeing then that these fourteen blended orders give a 
correlation with entrance order of 0-39, and the fourteen 
corresponding school orders also gave a correlation of 932, 
we ate safe in taking as our measure of the reljability of J's 
entrance examination the avetage of the twent -51х cortela- 
tions with School Certificate—namely, о4о. 

In order to afford a concrete idea of what sch a correla- 
. tion represents, I give Fig. IIT, which represents a correla- 
tion of 036 found between entrance orget (1924) and 
School Certificate order (1928) in a school in Centre J. 

In Fig. Ш a smaller proportion of boys nderwent large 
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changes: in order than was the case in Fig. П (p. 56), which 
teptese.nted a correlation very similar to the average for 
Centres, A, B, C, D, and E. Yet even in Fig. IIT great 






























divergelnces still appear. 

Top| group Middle group 
Order at Order in || Order at Order in | | Order at Order in 
entrance Sch.Cerk | | entrance Sch. Cert. | | entrance Sch. Cert. 

1 1 1 і 
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4 Tu 4 4 4 
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9 9 9 
1 10 10 о 
14 11 3 
12 12 2 
Y 13 13 5 
14 14 4 
Em. \ 18 DN 5 

16 16 

17 17 

18 18 

19 19 

20 20 

2f 2f 

22 22 

. i 25 23 





Fie. III. 


Note espetially : 
(i) Thé top boy at entrance drops to roth four years 
1, сег, and the 3rd to 18th. 
(it) Nod 12 at entrance rises to 2nd, while close by him 
at\entrance, No. 15 drops to bottom but one. 






whil@ No. 18 soats to 6th. 


| 
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Still, on the whole, Fig. III represents a decidedly greater 
degree of resemblance than did Fig. I1; and this exemplifies 
the fact that the average correlation is decidedly higher in 
Centre J than it was in Centres A, B, C, D, and Б. This 
higher correlation is also more reliably demonstrated than 
were the correlations in Centres Н and I. 

In Centre J, with the exception of one or two early yeats, 
the enttance examination included questions of a mental 
test type, and in the two most recent years the order of 
admission was based partly on papers of mental tests, and 
partly on papers in arithmetic and English. 


IMPROVEMENT OF EFFICIENCY OF ENTRANCE EXAMINATION 
OF CENTRE J IN LATER YEARS 

At the same time that mental tests were introduced in 
Centre J, certain other changes were made (e.g. more 
detailed standardisation of essay marking, etc.), and I 
thought it of interest to see if these changes were followed 
by higher correlations with later school work. 

Taking the 11 orders available (for classes of 18 pupils 
ot over) for the years of entry 1926-1927, we find an average 
‘correlation of entrance order with School Certificate of 0-46 
(the lowest being o-12 and the highest 0-68). For the pre- 
ceding three years the average of fifteen classes was 0°35 
(range orr to 0°58). (See Appendix I (e), р. 179, for 
details.) X 

Thus there is an indication of some increase in the 
cortelations following the use of the mentaltests and the 
other changes introduced in 1926. 

Yet even this higher correlation of 0-46 leaves us with 
marked divergences between the orders, as shown in 
Fig. IV, where we have the two orders in a school in 
Centre J giving a correlation of that size—viz. 0-46. 
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It will be seen at a glance from Fig. IV that even the 
correlation of o-46 is consistent with a few striking changes 
in order. Note especially : 

(1) That of the two boys bracketed 14th at entrance 
one tises to 3rd and the other drops to zoth. 


Middle group 


Order at Orderin||Orderat | Order in 
entrance Sch.Cert| | entrance Sch. Cert. 


j 
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(2) One boy bracketed 8th at entrance drops to the 
bottom of the class, while the 18th rises to sth. 
The average correlation of 0-46 for the most recent years 
of Centre J, however, suggests a definitely improved 
entrance examination, and one far above the centres A, B, 
C, D, and E, though of course they too may be now making 
improvements in their methods. 
Even in Centre J, however, there remains considerable 
divergence between the orders in the entrance examination 
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and the orders in the examination four or five years later. 
It has indeed been calculated that even if an admission test 
correlates with success at final performance to the extent of 
0*5, this means that one-third of the pupils admitted should 
have been replaced by pupils who failed in the admission 
test. In most of the other centres these divergences are 
even greater. In none is the resemblance so great as to 
assure us that the most deserving pupils are gaining scholar- 
ships ог free places, or that the examination is keeping out 
those unfitted for secondary education, or that it is not 
excluding some who ate so fitted and even more deserving 
of scholarship awards than are many who gain them. 

I should like to make it clear again that I am bringing no 
general charge of incompetence against those responsible 
for these examinations. In one or two cases I surmised 
from vatious indications that the examinations were planned 
and conducted in an amateurish way. In the majority 
of cases, however, I have no doubt that they are most 
carefully considered. Many such entrance examinations 
throughout the country are in the charge of an experienced 
examiner ; in two of the groups we have studied they were 
supplemented by mental tests and in some by an interview.? 

The centres I have discussed may not be a fair sample ; 
but personally I very much doubt if many are as good as 
the best here studied, though in a few more reliable intelli- 
gence tests may be in use. The plain fact is that at present 

1 See Dr. А. S. Otis, Statistical Method in Educational Measurement (р. 223), 
Chicago, 1926. The statement quoted is, of coutse, dependent upon conditions 
as to number of cases, normal distribution, etc. 

2 Useful information as to the variety of arrangements made for such admission 
examinations is given in a report on Examinations in Public Elementary Scboo!s by a 
Joint Committee of the Association of Education Committees and the National 
Union of Teachers (London, 1930); and also in the Board of Education pamphlet, 


No. 63, on Free Place Examinations (1928), in which a criticism of methods and 
organisation, based on a study of 57 examining authorities, is given. 
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there is no well-established agreement as to how best to 
select pupils for the type of education now usual in 
secondaty schools. 

This must not, however, be taken to imply that the solu- 
tion lies in restricting the number of entries into secondary 
schools. Our low correlations, together with the examples 
of the crowded border-lines given in Chapter ПІ and the 
subsequent success of many pupils near the bottom at 
entrance, indicate that there are still many who fail to gain 
admission who ate worthy of a secondaty education. It 
has been suggested to me that an economist in education 
may seize on some of our findings as evidence that we ought 
to spend less on education after 14 years of age. To such 
I would point out that, even if at present many of our 
awatds аге misplaced, the right alternative may be further 
education of a different kind from the present secondary 
school curriculum. Lastly, further research will undoubt- 
edly make our selection of pupils more efficient. 


Chapter VI 


CAUSES OF DISCREPANCIES BETWEEN ORDERS AT ENTRANCE 
TO SECONDARY SCHOOLS AND ORDERS FOUR YEARS LATER 


lr may be best to give first what seem to be partial causes of 
discrepancies between performance in the entrance exami- 
nation and performance four or five years later, and then 
to show that certain factors have not as much influence as 
is sometimes imagined. 

(1) One probable cause, in certain cases, is that the special 
coaching of some pupils for the entrance examination places 
them above their ‘true’ position relative to other pupils 
of equal innate abilities, who have not been so crammed. 
This explanation was stressed by several Headmasters 
with whom I discussed the problem. 

It may be said that special cramming at School Certificate 
stage may also upset the ‘true’ order of merit. But pres- 
sure and coaching specific for the examination may be more 
general then; while most children from the elementary 
schools ate allowed to take the entrance examination in 
their stride without any special preparation. 

(2) Apart from special coaching, the fact that the candi- 
dates in the entrance examination come from several (some- 
times many) different schools, where the standard of teach- 
ing and work may vary greatly, means that some pupils 
have an inevitable advantage over others in the entrance 
examination. When pupils get together in the same 
secondary school this advantage soon ceases to affect 
their relative positions, those who have actually greater 
ability but had not been prepared so well as others in 
the primary schools now coming mote to the top. The 
Headmaster of the secondary school in Centre B, for 
example, was able to point definitely to an elementary 

тї 
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school where the standard of work was low, and where 
there was no special preparation for the entrance examina- 
tion, and he was able to indicate a group of four boys from 
this school, who were very low down in the entrance 
‘examination, but did very well later on in the secondary 
school—one nearly bottom at entrance gaining first-class 
honours in the School Certificate in four years. 

That (1) and (2) ate not the sole causes of our discrepan- 
cies, however, is suggested by the fact that mental tests, 
when given alone, give little better prophecy of future: 
success. For mental tests, though partly dependent on 
previous education, are far less so than is success in the. 
English and arithmeticfexaminations. (For the value of 
mental tests in entrance examinations see Noz at end of 
this chapter.) 

(3) Apart from the two above-mentioned causes, the 
order of merit in the entrance examination may be unstable 
because of the various types of ‘ luck’ (in questions, varia- 
tions of ‘form,’ etc.) which we discussed in Chapter І. 
These factors no doubt vaty greatly from examination to 
examination, and we shall discuss the matter more fully 
later in the chapter. 

(4) A fourth possible cause of changes in order during 
the four years is that qualities of character, diligence, per- 
sistence, and the like may tell more and more as time goes 
on. No doubt these are already tested to some extent 
before the change over to the secondary school ; for in the 
first instance the slackest pupils will not usually be recom- 
mended by the Heads of their elementary schools for 
admission to the entrance examination. Secondly, apart 
from this elimination, qualities of character are already 
involved to some extent in the work required to pass the 
examination in English and arithmetic. It should, how- 
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ever, be noted that so far as success in the entrance examina- 
tion depends on innate abilities (as it does largely in good 
mental tests), rather than on acquired knowledge, so far 
such success will be independent of qualities of character. 
We shall show directly that qualities of character cannot 
be the sole cause of the changes in the order of merit. 

(5) The usual greater ‘scatter’ of marks in arithmetic 
in the entrance examination will give that subject an ex- 
cessive and possibly determining influence, unless pre- 
cautions ate taken to avoid this, as shown elsewhere 
(p. 177). i 
- (6) Lastly, and perhaps most important of all, the 
entrance examination in our centres was, as is usual, only 
in atithmetic and English—together with (in two centres) 
some mental tests. The School Certificate examinations 
and the schools? own examinations, on the other hand (even 
after only one year), include as a rule English literature, 
history, geography, algebra and geometry, French, some 
science, and possibly Latin. In most, if not all of these, 
specific mental capacities and interests are probably in- 
volved, which are not tested by the English and arithmetic 
tests in the entrance examination. 

Even if performance in the various school subjects 
depends chiefly upon general ability or abilities, no ex- 
регіепсеа secondary-school teacher will question the im- 
portance of specific abilities (or interests) for mathematics 
and foreign languages; one may have a pupil who is 
good in every branch of school studies except one. The 
same applies, though perhaps to a lesser degree, to the 
physical sciences, and sometimes apparently to history 
and geography, ‘interest’ perhaps being the greater 
factor here. | 

Thus, even if by good tests of general intelligence pupils 
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are well selected from the point of view of general ability, 
the specific abilities may be inadequately tested. 

We shall return to this question of specific abilities and 
interests. First, however, let us consider several other 
conceivable causes of the discrepancy between orders at 
entrance and four years later. 


Is OUR CRITERION, PERFORMANCE AFTER Four YEARS, 
ITSELF RELIABLE ? 


It may be suggested that the causes of discrepancy lie 
in the unreliability of the orders in the School Certificate 
examination and the schools’ own orders after four or five 
years. These аге of course not infallible ; but we have seen 
that they correlate very highly with one another2 Also, 
the School Certificate examination is usually based on six 
ot seven subjects—probably a dozen or more papers ; 
while the schools’ own examinations would usually add 
eight or ten more papers ; so that the element of luck is 
greatly reduced, when orders are based on the zoa} marks. 

Also, when combined these two examinations form a 
better test than either taken alone. Now, in all the schools 
(except one) for which we have been able to obtain blended 
orders (of School Certificate plus school orders) we have 
seen that these blended orders give no higher correlation 
with the entrance orders than do the School Certificate 
orders or the school orders taken alone. 


1 Dr. H. G. Stead gives evidence of the importance of both general and specific 
abilities for various groups of secondary-school subjects in the ‘Correlation of 
School Subjects,” Forum of Education, vol.i, 1923, p. 27. А mote recent and as yet 
unpublished research, by Mr. J. H. Wilson, suggests that success in the varied. 
subjects of the School Certificate depends more on specific abilities than has been 
thought by some psychologists. 

2 The average for the four centres, A, B, C, and E, was about o'8 (see Chapter IV, 
p. бт). The average for eight classes in Centre J, namely o-82, is given later in 
Table I (p. 87). ‘ 
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This is a notable result. We must surely believe that 
the blending of the orders does tend to give truer estimates 
of the relative capacities of the boys at the time (so far as 
they can be tested by examination) than does one alone. 

The fact that this more reliable order at School Certificate 
stage gives no higher correlation with entrance examination 
order, suggests again that it is not the unreliability of the 
School Certificate examination which results in such low 
correlations with entrance order; but rather that the ex- 
planation lies in the nature of the entrance examination, or 
in other causes. 

The discussion in Chapter I will have shown clearly that 
the present writer is far from believing that gaining, or 
failure to gain, a School Certificate is a completely reliable 
measure of a pupil’s worth, or even of his scholastic attain- 
ments; but to suggest that it is of no value as a test of 
pupils’ successes in the school course, especially in large 
numbers, is to adopt a sceptical attitude as to the value of 
examination success far beyond that adopted in this book. 
Also, we are here chiefly concerned with the inter-correla- 
tion of examinations. So far as examinations are reliable, 
they ought to agree with one another—that is, of course, if 
they profess to be general assessments of the students’ 
capacities as a whole. 

Furthermore, the criticism in Chapter I as to the School 
Certificate examination referred chiefly to the possible 
dependence of the award of the Certificate on a narrow 
margin of marks in one subject. The position taken in this 
enquiry is that the /o/z/ marks are a more reliable index, and 
that the combined marks of the School Certificate and the 
schools’ own examinations are more reliable still. 

Such a combination should compensate to some extent 


the pupil who, through ‘ bad form ’ on some day or through 
6 
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exceptional ‘bad luck" in the questions of one ot two 
papers, has not done himself justice either in the School 
Certificate or in the school’s own examination. For closely 
as the orders of these two examinations resemble one 
another, their high correlation is consistent with a few 
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marked divergences as is shown in Fig. V, which represents 
the performance in School Certificate and the schools’ own 
examinations, giving a correlation fairly typical for Centre J, 
namely o-88. 

It will be seen that there are one or two individual 
changes of considerable size—e.g. from 2nd to oth, and 
17th to той. Substantially, however, each group remains 
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in the same group for both examinations, and the greater 
resemblance of these two orders (giving a correlation of 
0:88) than that between school entrance and School Cer- 
tificate (with a correlation of about 0-36) may readily be 
seen by comparing Fig. V with Fig. III, p. 72. 


THE INFLUENCE OF QUALITIES OF CHARACTER; AND OF 
VARIATIONS IN RATE OF MENTAL DEVELOPMENT 


Everyone will agree that qualities of character affect the 
order of merit in school work—and one would expect such 
qualities to tell more and more as time passes—though 
some pupils have only temporary slack periods and then 
recover. Also we have agreed that some pupils are likely 
to shoot ahead of others through more rapid mental de- 
velopment ; it is already known that the clever young boys 
‘ina class tend to pass the older ones in the course of time. 

The influences of these two factors are hard to separate 
in our enquiry ; but we can test at least their joint effect. 
Leaving the entrance examination for the moment, we can 
ask—How much change is there within the school’s own orders 
over a period of, say, four orfive years ? This will give us 
a clue as to how fat change goes on with the mere passing 
of years. 

Evidence on this point was obtained first from a large 
high school for boys. ‘There I took the position in school 
of groups of boys between 12 and 13 years of age, after at 
least a year in the school to give them time to ‘settle 
down’ and get appropriate moves. These positions I 
compared with their positions four or even five years later. 
The results of four classes showed an average correlation 
of o-79 for four groups comprising 74 boys inall. (Details, 


Appendix I (g), p. 180.) 
Of course, this correlation of 0-79 still allows for con- 
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siderable changes in a few individuals—one forging ahead, 
another failing to fulfil his earlier promise ; but the average 
of these correlations in the high school, viz. o-79, is far 
higher than the average correlation between entrance and 
School Certificate examinations, although the interval (of 
fout ог five years) between the two stages tested is as great 
as that between entrance and the usual School Certificate 
stage at a secondary school. 

To gain further evidence on this question of changes in 
position due to different rates of development, I obtained 
from several schools in Centre J the order of merit after one 

Jear at school, in ordet to compare it with the order at School 
Certificate stage. Figures wete obtained from six schools 
(regarding the boys’ and girls’ divisions of a co-educational 
school as separate schools). 

The correlations (for fourteen classes of 18 "pupils or 
more in six schools) were obtained between the order after 
one year in school and the order in School Certificate (three 
or four years later), and the average of these was 0-69 for 
298 pupils. (See Appendix I (2), p. 181, for details.) 

Thus, there is a decidedly greater resemblance between 
(2) the order after one year in the secondary school, and the 
otder in the School Certificate examination three or four 
years later, than between (2) the entrance examination order 
and the order in the School Certificate, the average corre- 
lation fot which, for these fourteen classes, was 0°37. 

In other words, a considerable change in orders must 
occur in the first year. If it be replied that we are compar- 
ing a three-year period (first year to School Certificate at 
fous years, as it was in most cases) with a four-year period 
(entrance to School Certificate), the answer can be supplied 
with a further figure. We can compare the entrance order 
with the school order three years later. We find then no greater 
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resemblance, namely, a cottelation of 0-35 as against 0:67 
between first year order and School Certificate. 

That is, the changes (а) between the orders at entrance 
and three yeats later are nearly twice as great as the changes 
(b) between first year order and School Certificate also three 
(or even four) years later. 

In short, the big changes come in the first year in the secondary 
school, the first year of more varied studies ; and this finally 
is confirmed when we compare the order at entry with the 
order after one yeat in the secondary school. For nine 
classes (of the same group of fourteen just referred to) the 
figures were as follows : 

Correlation of entry order and first year order in 
secondary school, 0-43. 

It may be suggested, however, that the age allowance 
made in the entrance examination (but not in the first year 
order) would tell rather heavily in this comparison.? The 
younger boys have not yet had much time to make up for 
their deficiency for which the age allowance was made. 
They may be regarded as still needing some age allowance 
at the end of their first year. So let us compare the order 
at entry without age allowance with the order after one year 
in the secondary school. We find it is still only 0-46 (for 
eight of the classes, comprising 170 pupils). As after all 
the age allowance at entry professes to arrange the candi- 
dates according to their then fitness for secondary-school 
work, the former correlation (0-43) is probably the more 
relevant one. But suppose we take the average—say 0-44. 
Once again we have evidence that there is a more decided 
change between the entry and the end of the first year than 
there is between the first year and the fourth or even fifth 


1 Based on seven of the above-mentioned fourteen classes. 
2 The question of age allowance is discussed briefly in Appendix I (), p. 182. 
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yeat, the correlation for which was 0-69. Indeed, the 
cottelation between entry order and first year order is 
scarcely gteater than that between entry order and School 
Certificate order four or five yeats later. 

In other words, it is evident that changes with the mere 
passing of time are not the main factors in causing big 
changes in the orders of merit. It seems most likely that 
this greater change of otder, which occuts in the very fitst 
yeat, is due to the great change in the type of wotk done in 
the secondaty school as compared with that tested in the 
entrance examination. The first year examination would 
include at least one foreign language, mathematics, some 
science, history, and geography. 

Notice that we cannot attribute this greater resemblance 
of the first year order and the fourth year to the fact that 
they ate both orders within the school ; for we have compared 
the first year order with that of the public Schoo] Certifi- 


cate examination. 


CHANGES FROM YEAR TO YEAR IN THE SECONDARY 
SCHOOL 


In the last section we have shown that the changes in 
otder from entry to School Certificate stage cannot be 
entirely due to different rates of growth. There is a big 
change at once—bigger in the first year than in all the other 
three ot four years put together. 

Yet, as already indicated, there are undoubtedly some 
changes due to the factor of differential growth or to the 
increasing influence of character qualities. This is not the 
sole remaining cause, however, of the variation even simply 
within the school, from year to year; for there is the 
instability of the schools’ own yearly examinations to be 
remembered. 
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For a small group of classes I was able to get the orders 
for the same boys at entrance and after one year, two years, 
thtee yeats, and four years in the school. Thus we 
obtained the correlations between the orders for each year 
and the next. They were as follows : 


TABLE I 


CORRELATION OF ORDERS FROM YEAR TO YEAR IN THE SECONDARY SCHOOL 
(Based on average of eight classes—16g pupils) 


4th (or pth 
year schoo! 
Entrance order! у= year order | 2nd yearorder | 3rd year order order and order 
wi bed year | and 2nd year | and 3rd year | and 4th year | in School Certi- 
order ficate of the 





same year 
With age allow- 

ance : 0:43 — — — — 
Without аре 

allowance . 0:52 0:74 о'81 0:79 0'82 





It will be seen that the correlations of orders, separated 
by one year, within the secondary school—based on the 
schools’ own examinations in similar subjects—are them- 
selves on the average o:78—not much more than the cot- 
relation of first year order with School Certificate three or 
four years latet.1 Indeed, the correlation of the first year 
and second year orders, viz. 0-74, is practically the same as 
that between first year and School Certificate order three 
years later, viz. o-71 for the same group of classes (except 
one for which this figure was not obtained). In other 
words, the passage of two mote years does not appreciably 
lower the resemblance of the later order to the first year 
order. 

Roughly speaking, the variability of the schools’ „озуп 
orders from year to year is represented by a drop in cor- 


1 Та this average the fifth column is not of course included, as it involves an 
order in the School Certificate—not an order within the school merely. 
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relation from complete resemblance (1-0) to about o:8— 
that is, a drop of about o:2. 

The number of classes covered by Table I above is small, 
and they had to be taken from only four schools. They 
suggest, however, that the mere instability of the schools? 
Own examinations and the /ezorery variability of a boy 
from year to year are the main cause of the changes of 
orders within the school and not a steady continuous pulling 
ahead due to character qualities, or to continued and pro- 
gressive growth at different rates ; in spite of the fact that | 
undoubtedly the younger able boys tend to move ahead of 
the older less able ones. 


INFLUENCE ON SCHOOL ORDERS OF TEACHERS’ 
KNOWLEDGE OF PUPILS 


The question may be taised as to how far the high cor- 
relation of one school order with another is due to the fact 
that masters are influenced in their marking of examination. 
answers by their knowledge of the boys. Such knowledge 
may have a steadying effect on the marking of the same boy’s 
work at different times. A boy who has once been judged 
vety good will tend to be marked high, it is suggested. 
This has been ‘called the ‘halo’ effect on the marking of 
answets. I have mentioned elsewhere this undoubted 
possibility. In the case of the orders dealt with in Table I 
(p. 87), however, such influences could only be very slight; 
for in all these schools there was a common examination 
each year for all the four parallel classes ; and in the exami- 
nation, one master marked one or two questions through- 
out the papers in a given subject, and would only know one 
section of the pupils, while another master would mark 
another question, and so on. 

Furthermore, that ‘halo’ effect cannot be an important 
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factor in causing high correlation from year to year is shown 
by the fact that the correlation of first year school order 
with the order of the School Certificate examination 
(marked by external examiners) was actually as high 
(0°63) as the correlation of first year with fourth year 
school order (0°62) for the five of the above-mentioned 
classes (115 pupils) for whom these correlations were 
obtained. 

On the whole, it seems clear that the relatively high 
degree of resemblance of school orders from year to year, 
in the examples we have just considered, is not due ap- 
preciably, if at all, to any such * halo ' effect. 

To resume our main argument, we may point out that 
the order of merit after one year in a secondary school is 
a moderately good prophecy of attainment after four or 
five years (represented by a correlation of o*69 in the 
secondary schools and o:8 in the high school). True, the 
differences between the orders at one year and the orders 
'at four years leave toom for considerable advance or re- 
gression in individual cases ; but at least the order of merit 
after one year is a better guide as to where a boy will be 
three or four years later than the entrance examination is 
as to where he will be one year later. 

From all lines of evidence then it seems unlikely that 
different rates of development, or the effect of qualities of 
character, are the most important causes of changes in order 
of merit from entrance to fourth year, unless we suppose 
that such influences occur almost entirely during the first 
year in the secondary school, a most improbable supposition. 
We may also point out that no improvement in the entrance 
"examination could overcome irregularities due to* the 
development of character. So far as such factors exist, 
they are reasons for not regarding it as settled at 11 + years 
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of age once and for all whether a child: shall have further 
education beyond the age of fourteen. 

The large immediate changes in order duting the fitst year 
in the secondary school (neatly twice as great as the changes 
in the next three yeats at the school) point, however, to 
some further cause.! 


INSTABILITY OF THE ENTRANCE EXAMINATION AS A CAUSE 
OF DIFFERENCES BETWEEN ORDERS AT ENTRANCE 
AND Four Years LATER 


We have still to consider mote carefully the mete in- 
stability of the enttance examination test, instability in the 
sense that a repetition of a similar test (but with different 
questions) on the next day might give different results. 
This is the type of unreliability due to (a) the various kinds 
of * luck’ referred to in Chapter І, the special liking or dis- 
liking of the particular questions set, the specially good or 
bad ‘form’ of the examinee on the particular day, etc.j 
ot to (7) variability in marking by different examiners. 

Of course, stability is no guarantee of the efficiency of 
the examination. For a similat ordet may be given by two 
examination papers, which is nevertheless a bad order for 
the putpose of selecting the pupils ; but if a vety different 
ordet is given on a repetition of the examination, obviously 
we Cannot trust the examination. 

Such instability is almost certainly one factor at work. 
We have seen that the School Certificate order, based on 
about ten or a dozen papets, correlates with a similar 
examination in the schools at about the same time to the 
extent of o-8—a drop of o-2 from unity. One may fairly 

1 Н. С. Stead has offered some evidence of change in a chatactet-quality affecting 


school work. See “Factors in Mental and Scholastic Ability," Brit, Journ. of 
Psychology, vol. xvi, (1926), p. 199. 


Chap. у] INSTABILITY OF ENTRY EXAM. 9I 


assutne that the instability of the entrance examination in 
only arithmetic and English—perhaps only two and at the 
most fout papers—is at any rate greater than this. When, 
however, mental tests form a substantial part of the 
examination, such instability should be considerably 
reduced.* 

We have already seen that if an essay fotms an important 
patt of the English paper, the mere instability of marking 
may be great, though it may be greatly reduced by adopting 
a systematic scheme for marking.? Instability, due to vary- 
ing suitability of the essay topic, or to varying ‘form’ of 
the pupils, is another factor. Few centres, I imagine, are 
likely to rely on the essay alone; though in one of the 
three centres whose papers I saw, the English paper con- 
sisted entirely of essays; and only two topics wete given, 
both of which had to be attempted. With such a paper 
instability would surely be very great. (The average 
cottelation of the essay paper in one School Certificate 
‘examination with the order of merit (in composition) given 
by the schools is only about 0-41.) 

Where the English papers of the entrance examination 
include different types of questions, the instability of the 
order of merit will vary with the type of question. If many 
questions are set involving special knowledge (e.g. of gram- 
matical rules), much will depend obviously on whether 
the appropriate instruction has been given: pupils from 
one school might do well,on one paper and badly on 
another. In such papers, too, general intelligence may not 
be so important as knowledge of particular facts. Again, 

1 We know that the best types of intelligence tests give, when repeated, orders 
correlating as highly as о-у. Опе would not, however, expect so high a correlation 
as this with the se/ecfed group chosen for the entrance examination. (See note on 


the influence of selection, Appendix I (j), p. 184.) 
2 See Chapter П, p. 26. 
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if answers, even short ones, ate of the essay type, the per- 
sonal factor of the marker may tell heavily. 

Instability due to such factors may be a very important 
cause of the prognostic unreliability of the entrance exami- 
nations in some of the centres we have considered. In the 
entrance examination for one centre, for example, the order 
of merit in English (for 65 boys admitted to the main 
secondary school) showed no correlation with the order in 
arithmetic—a striking result, suggesting that one paper 
at least hardly depended to any marked degree on general 
intelligence as contrasted with special knowledge and train- 
ing. Indeed, the Headmaster of the school concerned said 
that both papers were such as to call rather for special 
training or information rather than for innate ability.t 

I had evidence, however, that English papers of the 
type sometimes set in these entrance examinations may be 
of a high degree of ‘ stability.’ At my request the English 
papets previously set for one county entrance examination 
in 1930 wete given to a class of 4o pupils (of тт and 12 
years), and the order of merit on the combined papers was 
drawn up. A week later the two papers in English set for 
the same examination in 1931 were given to the same class 
and another order of merit obtained. The two orders 
cottelated as highly as o*9, when teacher A was the exami- 
net, and o:95 when teacher B was the examiner—cortelations 
compatable with those given by the repetition of a good 
intelligence test. Here a detailed scheme of marking the 


* Previous thorough * selection ’ of pupils of such a nature that all examined were 
about the same level in general intelligence (or in any ability largely involved in both 
arithmetic and English) would explain such absence of correlation. And some 
degree of selection of this type there usually is in the first selecting of pupils who 
shall be candidates for the entrance examination. But there was no reason to 
suppose any such thorough previous © selection ? as would so level all the 65 boys 
entering the school. (See Appendix I (j), p. 184, for a note on the influence of 
selection.) 
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English papets was agreed upon ; and the question papers 
were in my opinion, very good ones. 

Elementary arithmetic papers may be expected to corre- 
late at least as highly with one another. One of my 
advanced students set a paper of ten questions in arithmetic 
(of a type usual in entrance examinations) to his class of 33 
pupils ; a paper of ten similar questions was set a few days 
later, both being marked by the same teacher. The two 
orders correlated to the extent of 093. 

Where, as above, the marker of the papers knows the 
pupils, it is possible (as already indicated) that this know- 
ledge has a steadying influence on the marks which may 
increase the correlation, at least in English. The teacher 
knowing that Jones is aswally good (or weak) in English 
may unconsciously err on the generous (or severe) side 
when Jones writes a specially weak (or good) answer in one 
paper. With an external examiner this steadying influence 
would not be present, so that such correlations might tend 
to be somewhat lower. But with carefully arranged bases 
of marking, such influences should be small. 

A more serious factor in the big public entrance examina- 
tions is that the numbers of pupils are so large that the 
markers of the answers may find difficulty in keeping their 
own standard the same throughout. This is balanced to 
some extent by the checking of the chief examiner, but he 
cannot re-mark every script, so that occasional lapses may 
be missed; and further, even the chief examiner's own 
standard is likely to oscillate. 

It would, however, be of great value if some local 
authorities could arrange for a similar set of papers to those 
set in their secondary school entrance examination to be 
done by the same pupils in their schools with similar re- 
strictions as to time, etc., the week after the entrance 
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examination, in order to see how far the repeated test gave 
a similar order of merit; or at least a ' repeat ' examination 
might be given to a large section of border-line pupils— 
just above and just below the admission mark—and this 
second examination might be used to make the final 
selection. 

As pointed out already, the ‘ stability’ of the entrance 
examination is likely to be greater when good mental tests 
ate used: and we have seen that in two of the centres we 
have dealt with mental tests wete used ; and their introduc- 
tion was followed (in the only centre where it could be 
tested) by a raising of the correlation of entrance examina- 
tion and fourth year order. 

On the whole, while we should, I think, regard the 
* instability? of the entrance examination as an important 
cause in some centres of the low correlation with order of 
‘merit in School Certificate four ог five years later, we cannot 
regard it as necessarily so great (when the examination is 
well managed and includes mental tests) as to be the sole 
ot even chief cause of the great change in order of merit 
even in the first year. 


Summing up this chapter as to the causes of the great 
discrepancy between orders at entrance to the secondary 
school and orders four years later, we may conclude 
tentatively (for the subject needs further detailed enquiry) 
that the chief causes are those mentioned at the beginning 
of the chapter, namely :. 

(1) The special cramming of certain pupils, putting 
them above their true position in the entrance examination. 

(2) The supetior teaching some pupils obtain in certain 
elementary schools as compated with that in others. 

(3) The inherent inability of an examination in arithmetic 
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and English (and possibly even of good tests of general 
intelligence) to indicate relative ability for (or latent 
capacity for interest in) the many varied subjects studied in 
the secondary school. The entrance examination is further 
limited in this respect if the usual greater scatter of the 
matks in arithmetic papers than in English is not corrected. 

(4) Different rates of mental development and the 
gradual influence of good or bad qualities of character.. 
These undoubtedly tell in individual cases; but they do 
not seem to be as widespread as some have thought, and 
they cannot account for all or nearly all the changes in 
order of metit. 

(5) The ‘instability’ of the entrance examination—in 
the sense that a similar examination a few days later might 
give very different results at least within a considerable 
range of the critical border-line. This is probably an 
important cause in some centres, 


PRACTICAL SUGGESTIONS BASED ON THE FINDINGS OF 
Part П oF THIS ENQUIRY 


These results, I suggest, indicate first the need for 
further research and experiment in the mode of admission 
to secondary schools. "The extension of the use of the best 
type of intelligence tests is desirable on the ground that 
success in them depends much less on special previous 
education, and also because their stability is usually greater 
than that of examinations in*English or arithmetic. 

Some types of general intelligence tests, however, 
though a valuable help, do not ensure satisfactory selection, 
if we judge by the evidence so far available. Probably 
tests of specific abilities needed in various secondary-school 


1 The main evidence on the value of such tests is summarised in the Noze at the 
end of this chapter. 
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subjects (especially languages, mathematics, and science), 
which are already being devised and used, may eventually 
prove of further help. For example, such tests of ability 
fot French (needing only 45 minutes) have been found to 
cottelate with first term's work in French to the extent of 
about o:8 ; while another set of tests also gave a correlation 
with first term's Latin of o-8.1 

Possibly better guidance may be obtained by more careful 
noting of the recommendations of the primary school heads, 
and ‘weighting’ recommendations of those heads who 
prove to be most reliable, where cases are followed up. In 
Centre J the pupils not recommended by the primary heads 
usually do badly in the entrance examinations. A consider- 
able number, however, succeed in passing the minimum 
standard required and in beating (in the entrance examina- 
tion) some of those who are recommended. But among 
the small number of cases, namely fifteen, I have been able 
to examine in one centte, eight did badly (four leaving 
because of bad reports), only one obtained School Certifi- 
cate in five years, and one other was also taking the 
examination in his fifth year. | 

In two enquiries in the United States of Ametica it has 
been found that the records of work in the elementary ` 
school give a better prophecy of success in the high school 
than do intelligence tests—the best correlations being as 
high as 0-64 and o:79? In some instances, however, the 


1 See, for example, Prognosis Tests in Modern Foreign Languages (New York, 1929). 
PP. 24 and 25, and other publications of the American and Canadian Com- 
mittee on Modern Languages, especially 44 Summary of Reports on Modern Foreign 
Languages, by R. Н. Fife (New York, 1931). Also “ Psychological Tests of 
Mathematical Ability," by Leslie Fouracre, Forum of Education, vol. iv, 1926, p. 201 ; 
and * Investigation of Mathematical Ability in the Class Room," by W. S. Flack, 
Forum of Education, vol. iv, 1926. 

2 See the account of the enquiries by C. C. Ross and T. L. Kelly given by P. M. 
Symonds in Measurement in Secondary Education (New York, 1927), especially p. 399. 
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tions for secondary schools in England. | Such tests are | 


being used in an increasing number of centtes. In 1929 
twenty-two county boroughs and twenty-three county | 
authorities included mental tests in some form in the 1 
entrance examinations at 11 +.? 

Of the Education Authotities who have used intelligence 
tests for the purpose, a few have followed up the pupils 
to see how they fared in the secondary schools and how 
reliable the tests were. 

At Bradford it was found that the tests gave a тоге fe- 
liable prophecy of school marks than did the entrance 
examination. Other and more thorough investigations 
agree on the whole in finding that a combination of intelli- 
gence tests plus the wtitten entrance examination gives the 
best forecast of what the pupil will do, not only in his first 
and second, but even in his fourth year at school. | 

In connection with my own enquiry I also had figured © 
supplied by one Headmaster who tested his scholars onf 
entrance with Spearman's intelligence tests, and found the) 
gave (on the average of four classes) a decidedly bette 
prophecy than did the entrance examination (for Centre G) , 



















1 Only applications of tests for this specific purpose can be dealt with here. The 
general problem of the relation between general intelligence and scholastic achieve- 
ment is a very wide one, with a considerable literature, especially that based on 
researches in the United States of America. A recent work, which includes 
summary of much of this work, is Educational Achievement in Relation dol | 
Intelligence, by С. W. St. John (Harvard Studies in Education, vol. 15, Cambridge, f 
U.S.A.). Я 

2 See p. 189 of Report оп Examinatiths in Public Elementary Schools (London, 
1930). 

3 See Board of Education Report on Psychological Tests of Educable Capacity, 
1924, p. 152. 

_ 4 See especially A. Donald Amos, “ Examination and Intelligence-test Forecasts | 
of School Achievement," Brit. Journ. of Educational Psychology, 1931, vol. i, р. 73- у 
(In the two correlations with fourth yeat work, however, the mental tests proved 
as reliable as the combined tests and entrance examinations.) See also D, W. Oates, 
** Predicting School Achievement,” Journ. of Education, August 1929. 
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It may be said that the examinations ate to some extent a 
test of chatacter—of perseverance or of interest in work, 
It is for this reason, perhaps most of all, that their retentio | 
at present as a means of selection at 11 + is NI 

"Though it should be noted that tests are beginning to invade 
this province: Dr. D. W. Oates, for example, has found’ 
that certain tests of Will and Temperament, cottelated) 
with school order fout years later even slightly higher than 
did the combined entrance examinations in English and 
atithmetic.* 








1 The coefficients were respectively 0-42 and 0-40, Op. cita, p. 9. 


PART HI 


THE RELIABILITY OF THE AWARD OF ENTRANCE 
SCHOLARSHIPS AT THE UNIVERSITIES 





тот 


Chapter VII 


THE RELIABILITY OF ENTRANCE SCHOLARSHIP AWARDS AT 
PROVINCIAL UNIVERSITIES 


In view of the large numbers of scholarships now given at 
provincial universities, the reliability of their award is a 
question of wide interest. It has been estimated that over 
40 per cent. of the students in all our British universities ate 
in receipt of some emoluments, Oxford and Cambridge 
having a somewhat lower, and the other universities a 
higher, percentage than 4o per cent. But these apparently 
include students receiving grants from the Board of Educa- 
tion in order to prepare for the teaching profession. In 

e present enquiry these latter have been omitted from the 
main tesults, as such students are not selected merely on the 
basis of public examinations: it is true that their perform- 
ances in the Higher School Certificate and other examina- 
tions ate taken into account, but personality and the report 
of the Head of the school are also considered, and this 
process of selection seems to be justified, as will be shown 
latet. 

By far the greater proportion of university scholarships 
ate entrance scholarships, and we shall confine ourselves 
here to these. 

For my own enquiry I first obtained figures from five 
provincial universities in England and Wales (one being a 
University College in Wales) over periods of three or four 
yeats, except for one university, which only furnished 
particulars for one year. ‘The periods cover degree awards 
from 1925 to 1929. . 

'This gave me a grand total of 2,088 students, of whom 
864 were scholarship holders and 1,224 were non-scholars— 


1 Ernest Barker, Universities in Great Britain, p. 54. 
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the proportion of scholars being thus about 41 per cent. 
The scholarships included all those awarded by the universi- 
ties themselves and by county or botough authotities 
(usually on the results of the Higher School Certificate) and 
a number of special school scholarships. (A few scholar- 
ships tied to special schools were omitted from the results 
of one or two of the universities.) 

The tables below show the performance of entrance 
Scholars and Non-scholats in the university degree exami- 
nations in Arts and Science ; Law, Commerce, and Medi- 
cine were omitted as introducing complications. 

First let us take the performances of all Scholars in Arts 
and Science. 





TABLE II 
PERFORMANCE OF ENTRANCE SCHOLARS IN FINAL DEGREE EXAMINATIONS 

No. of Students Percentage 

1st-class honours > s 237 274 
2nd-class honours . д 299 34:6 
3td-class honours : 61 Ti 

Pass degree $^ эуе . 221 25:6 

Fail . А . . . 46 53 

864 100°0 


Tt should be made clear that in the universities third-; 
class honouts are not regarded as a very creditable perform- 
ance. It is no guarantee of general ability higher than that 
needed for a pass degtee, though it implies more special- 
ised studies. My own experience, in the Education De- 
partments of two universities, has made me welcome into 
the post-graduate course a pass graduate at least as readily 
as one with third-class honours; for sometimes other 
reasons than lack of ability have led a student to be content 
with a pass course. Even leaving on one side those with 
third-class honours, we see that 31 per cent., ot nearly one- 


* Details for the separate universities are given in Appendix II (a), p. 188. 
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line, the reliability of awatds is extremely low, and that 
many students ate entering the modern universities with 
scholarships much more deserved by other students. 

The superiority of Scholars to Non-scholars would be 
still less if we included among the Non-scholars students 
who are in Education Departments, and who were left out 
of our previous reckoning. This will be seen from the 
footnote below. 


1 THE PERFORMANCE OF STUDENTS IN UNIVERSITY EpucATION DEPARTMENTS 


In the great majority of cases, students in Education Departments are пс 
scholatship holders ; fot example, out of 238 students in three university educatio 
departments, only 48, about one-fifth, held scholarships. [At University A, out « 
59 students in the education department 18 obtained a scholarship to the universit 
At University B, out of 111 students rx had scholarships. At University C, out « 
68 students 19 had scholarships.] 

I obtained particulars as to the degree results of 390 Education Departmen 
students in one Welsh and four English university centres for two years. Th 
figures were as follows: 





TABLE IV 
PERFORMANCE OF STUDENTS IN EDUCATION DEPARTMENTS IN DEGREE 
EXAMINATIONS 
No. of Students Percentage 
1st-class honours . . 44 115 
2nd-class honours . . 192 ' 492 
3td-class honours E . 46 ir8 
Pass degrees . . . 108 277 
Total . . 7 ; 390 1000 


As these figures were gathered from students in post-graduate courses, those whe 
had failed in the degree were already ruled out. Therefore, in comparing these 
results with those of Scholars, we must also leave on one side the failures amongr 
the Scholars. We then find that, while a higher percentage of Scholars get ‘ fitsts,', 
the percentage of students getting only a ‘third’ or a ‘ pass? is about the same 
among Scholars and Education Department students. A much higher percentage 
of Education Department students get first- or second-class honours than is the 
case with Non-scholars outside Education Departments (about 61 per cent. against 
40 per cent., failures being ignored in both cases), a superiority too great to be 
explained by the fact that one-fifth of the Education Department students are also. 
Scholars. 

Tf, then, we had included Education Department students in our general statistics 
the superiority of Scholars to Non-scholats would have been even less than it was! 
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It occurted to me that some students who would have 
competed for and perhaps gained scholarships might have 
been ruled out because their parents had an income above 
some given minimum, and so were not able to claim 
financial need. І | 

With this in view І made enquiries of the five universities 
concerned, and also of a representative sample of six 
county education authorities and six city authorities in the 
immediate neighbourhood of one or other of the five 
universities. The replies show no such restriction for the 
awatd of scholarships, though the amount of the extra 
maintenance allowances may and usually does vary with 
the income of the parents. We may thus rule out the 
possibility of the performance of Non-scholars being in any 
appreciable degree improved by the inclusion, among 
Non-scholars, of able students refused scholarships because 
of the amount of their parents’ incomes. 

It is possible, however, that a few of the Non-scholats 
who do very well are students who did not compete for 
scholarships—though such numbers are likely to be small, 
in view of the fact that so large a proportion of university 
students, entering the Faculties of Arts or Science, do so 
with a Higher School Certificate on which such university 
scholarships are usually awarded. In any case, they do not 
affect the fact that so many of the Scholars do so badly. 

Furthermore, so fat as such students exist who would: 
have welcomed both the value and the honour of a scholat- 
ship, they indicate a weakness in the general system of 
scholarship awards—and an argument in favour of the 
inctease in the number of internal scholatships—which 
will be recommended latet. 


1 The only city that reported such a limit for scholarships had rejected only one 
applicant on financial grounds in the preceding five years. 
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SCHOLARSHIPS AWARDED DIRECTLY BY THE UNIVERSITIES 


So fat we have consideted the performance of all types 
of Scholars, including those elected dírectly by local autho- 
rities and a few by schools. The performance of many of 
these was so poor that I tried to find how far the universities 

‘themselves were responsible for the selection of such 
inferior material. With this in view I considered separately 
those Scholars chosen directly by the universities them- 
selves. It proved that these university Scholars proper did 
much better at their universities than did the other Scholars. 
The figures were as follows : 











TABLE V 
University Scholars proper. Other Scholars 
ыж з 5з ч е шы 

No. Percentage No. Percentage 
1st-class honours . : 94 45:6 143 22-1 
and-class honours , К 80 37'0 219 33:8 
3td-class honours, : 16 T4 45 6-9 
Pass degree. . А 2I 97 200 30°9 
Fail or withdraw Я . 5 23 41 6-3 

216 1000 648 1000 











Here are some striking differences. Twice as many 
university Scholars as other Scholars gain first-class 
honours ; while the number of university Scholars gaining 
only a pass degree is barely то per cent., against 31 per cent. 
for other Scholars. Among, the latter, failures ate also 
much more frequent. 

The poor performance of Scholars, then, is due chiefly to 
those who were awarded their scholarships by bodies other 
than the universities themselves : 44 per cent. of these, ot 
nearly one-half, fail to justify the award of a scholarship, 
and are beaten by one-third of the Non-scholars. 
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Even among the university Scholars proper, nearly one- 
fifth gain only third-class honouts, a pass degree, or none 
at all, and this group is beaten by one-third of the Non- 
scholars—and by three-fifths of students in education 
departments. 


Тнк RELATIVE PERFORMANCE OF MEN AND WOMEN 
SCHOLARS 


A further analysis was made of the degree results 
obtained by men and women respectively who held 
Scholarships awatded directly by the universities. The 
figures ate shown in Table VI. To increase the numbers 
Itook the awards made in the five selected universities ovet 
a period of six or seven yeats for each—for the years of 
entry from 1921 to 1926 or 1927 inclusive. "These awards, 
and the results gained by the Scholars in the degree exami- 
nations, were gathered from the calendars of the respective 
universities, together with information kindly supplied by 
Registrars. 

TABLE VI 
RELATIVE PERFORMANCE OF MEN AND Women University SCHOLARS 





Women 











No. | Мо. Percentage 
| 
1st-class honours 9 2 197 47 260 
and-class honours. n 122 тот 558 
3rd-class honours. B 19 12 66 
Pass degree. . . 36 17 9'4 
Fail or withdraw . А 20 | 4 2°2 
| ке сы 
Totals . 4 394 | 181 100°0 








Tt will be seen that, in proportion, nearly twice as many 
men gain firsts as do women. ‘The separation of the women 
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leaves the men Scholars with an appreciably better record 
than that of all university Scholars taken together, but we 
still find nearly one-fifth of the university men Scholars 
with a third-class, a pass degree, or none at all. 

When the results are analysed under Arts and Science 
separately, as in Table VII, we see that the poorer record of 
the women is not due simply to inferior work in Science. 


TABLE VII 


RELATIVE PERFORMANCE OF MEN AND Women UNIVERSITY SCHOLARS IN ARTS 
AND SCIENCE 





ARTS ScigxcE 
Men Women Men Women 


No. |Percentage| No. |Percentage| No. |Percentage| No. Percentage 




















1st-class honours 38 442 34 | 241 | 159] 5yz 13 36:2 
znd-class honours | 39 | 45-4 $9 | 6531 83 | 288 12 | 335 
rd-class honouts 4 46 8 $7 15 5'2 4| тт 
ass degree . 5 y8 хо TY 31| 10:8 1 194 
Totals ‚| 86 | 1000 | 141 | ооо | 288 | 1000 36 | тосо 





(The relative number of failures cannot be given for this group of Scholars, 
as the Calendars do not reveal in which Faculty the Scholars ate who fail.) 


The number of women Scholats taking science is 
vety small for comparison with the men; but it will be 
seen that in Arts, the superiority of the men in the gaining of 
jirsts is even greater than it is in Science. 


THe RELATIVE PERFORMANCE OF SCHOLARS IN ARTS 
AND IN SCIENCE 


Let us now see if there is any difference in the apparent 
reliability of scholarship awards in Arts as compated with 
Science. These are shown separately in Table УШ, men 
and women being counted together. 
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exact way of determining the relative reliability of scholar- 
ship awatds in Science and Arts. 

We can get a mote reliable estimate of this by the 
following method : 


To a student who gains ist-class hons. we awatd the 
position mark of 1. 

To a student who gains 2nd-class hons. we award thi 
position mark of 2. | 

То a student who gains std- class hons. we award 
position mark of 3. 

lo a student who gains a pass degtee we awatd 
position mark of 3. 

To a student who fails, we award the position mark of 


We can then find the average of the position marks of 
all the Scholars in Arts and the average of the positio’ 
marks of all the Non-scholars in Arts. Evidently 
smaller the mark the better the students have done, e.g. 
if all scholars got first-class honours the average matk } 
would Бет. If we adopt this method we get the followin f 
results, 


AVERAGE CLASS GAINED 


Arts Science 
Scholats . с : : 5 о лу 2:24 
Non-scholats — . . . : 1279 3:08 


ае See ETC 


It will be seen that the average ‘class’ of the Science 
Scholar is about the same as that of the Arts Scholar; ; 


appreciably poorer than that of the Non-scholat in Arts, | 
chiefly because of the higher percentage of failures and ` 
pass degtees in Science than in Arts. "Ug 

The supetiority of the Scholars to the Non-scholars й 
Arts can now be measured by the ratio of the average clas | 
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But to compate the results of Arts Scholars (or Non- 
scholars) and Science Scholars (or Non-scholars) directly 
we must beat in mind the difference in the frequency of 
high awatds in Arts and Science. 










































TABLE IX 
Awarps OF Honours DEGREES ONLY IN ARTS AND SCIENCE 
ARTS SCIENCE 
Class 
Total Percentage Total Percentage 
Ist 75 28:5 164 48-1 
Scholats 2nd 154 60:2 . 145 42:5 
3rd 29 II'3 32 94 
Totals : 256 100:0 341 1000 | 
Ist 28 10'9 73 29:6 
Non-scholars 2nd 187 12:8 118 48:0 
зга 42 16:3 55 22:4 
"Totals 5 257 тоого 246 1000 
Schol Я Ist IOI 19'7 237 40:4. 
ИО Е Е. 
р 3rd 71 I3:9 87 14:8 
Totals ; 513 100'0 587 100'0 








Table IX gives the percentage of awards (fitst-, second- 
and third-class) among honours students only. This appeats t 
confirm the impression, not uncommon in the provincia 
universities, that Science is more generous than Arts i 
its awards of firsts. It is seen to be so here for both 
Scholats and Non-scholats. Among all students gaining 
honours degrees in the Science faculties, до per cent. 
gain first-class against 20 per cent. in Arts. 

The figures as given in Table IX, however, must not be 
taken alone. For it is possible, and it seems to be the cas 
in some univetsities, that thete is a more general weedin 
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out of candidates for ап honouts course in Science than 
there is for Arts. ‘hus, our earlier Table УШ (p. 111) 
showed that, among both Scholars and Non-scholats, 
a relatively greater number of Science students ate con- 
“ned to pass courses than is the case with Arts. 
Table X below summatises the figures for both Scholars 
id Non-scholats, and shows what proportion of al 
udents in Arts and a// students in Science gain firsts, 
conds, etc. 

















TABLE X 
COMPARATIVE PERFORMANCES OF all STUDENTS IN ARTS AND SCIENCE 
ARTS SCIENCE 
Total Percentage Total Percentage 
-class honouts о : тот 12:6 237 18:5 
l-class honouts . 2 341 42:4 265 20:5 
d-class honouts . : 71 8:8 87 6:8 
sdegree . А 5 225 28:0 510 397 
| È : б 66 8:2 187 1415 
Totals : 2 : 804 _ I00'0 1,284 IOO'O 














From Table X it will be seen that, if we assume that en 

asse the Science students ate as good as but no better 
an in Arts, then the examiners in Science ate mote 
merous in the awards of firsts, but more severe in the 
watd of seconds, than is the case with Arts. 

To be more exact, the proportion gaining fitsts in 
science is half as big again as the proportion gaining firsts 
m Arts; but the proportion gaining only a pass ot fail in 
Science is also half as big again as in Arts. : 

Again, if we group ‘firsts’ and ‘ seconds’ together : as 

zood honouts,’ 55 pet cent. of the students in Arts gain 
ch good sonore degtees, against only 39 per cent. of 
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the Science students. If we calculate the average class | 
gained we find the following : 


Average class gained by all students in arts . о : я = 2:48 
Average class gained by all students in science 5 т Я s le 2na 

















On the whole, then, Arts students gain slightly high’ 
awatds. | | 
The subject is a complicated one, with different facto 
relevant in different universities. It i$ for each universi 
to consider its own results in the light of its own regulation 


THE RELATIVE DIFFICULTIES OF GAINING “FIRSTS 
IN DIFFERENT SUBJECTS 


The problem exists, not merely between Arts a 
Science, but between subject and subject within ea 
Faculty. Ina given university and among the neighbou 
ing schools supplying most of its students, a rumour m 
get abroad, deservedly or not, that it is easier to get a 
“first” in one subject than another. Mere percentage 
cannot decide for us whether such is, ot is not, the cas 
For a gteater proportion of Firsts in Subject A than i 
Subject B may be due to one or mote of several causes : 

(т) The students of A may include a larger proportio 
of gteat ability in their subject than is the case with Subje 


O The teaching in Subject A may be more effective th 
: that in Subject В. : 

(3) The examiners in Subject A may be on the averag 
mote generous in the awards of first-class honours than 
those in Subject B. | 

(4) Subject B may be reputed to be an easy subject, ani 
so attract a large number of weak students, who lower 
proportion of firsts. 
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It is no doubt difficult to say what exactly should be 
involved in the award of first-class honours ; but broadly 
interpreted, explanations (1) and (2) above will hatdly be 
accepted by many as accounting for the difference between 
first-class honours awards in Arts and Science. 

It is also extremely difficult to discuss thoroughly what 
an be meant by the equality of ability in different subjects 
xt the frequency of high ability in different subjects. What 
an an “equal degree of ability ’ for chemistry and French 
ican? If different specific abilities are involved, how can 
re say that one is equal to the other, unless perhaps there 
i some reference to the frequency of a high degree of the 
becific abilities for chemistry or French found in the 
ommunity as a whole? Or does it refer to comparative 
neral ability plus some reference to frequency of the 
»edific abilities? We do speak of a man as one of 
exceptional ability’; and unless we do compare two 

students in different subjects by reference to general 
ability of a type common to both, it would seem we tust 
all back upon the rarity of the specific ability shown. 

Meanwhile, the superficial judgment of the strictness of 
tandatd of an honours school by the general impression of 
eachers of other subjects is dangerous. For we do not 
now the extent to which general or specific factors are in- 
olved in a given subject. 

In a post-graduate Education Department one has good 
ypportunities of observing in an individual student the 

vatiability of capacity fot different subjects. Ihave known 
students with a very low honours, ot a pass degree, reveal 
a psychological ability, judged by the external and internal 
(examiners, to be of first-class honours standard; on the 
ther hand, I have known students with firsts or secon in 
atious subjects—Atts ot Science—who would apparently 


have only gained a third-class if they had taken an honours 
course in philosophy or psychology. ; 

Complex as the problem is, it is undoubtedly highly. 
desirable that, so far as possible, a *first? should mean a 
similar standard in different subjects ; otherwise a specus s 
of ‘unreliability’ attaches to the results of the examin, 
tions, and they are liable to be misunderstood, not only b - 
the general public, but even by those closely associate, 
with the administration of education and the selection 4 
teachers. n 


|.| 
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{ 
CONCLUSIONS AS TO SCHOLARSHIP AWARDS IN SS 
PROVINCIAL UNIVERSITIES: \| 


| 

Summing up this chapter we may say that the facts а 
corded earlier as to the poor performance of over опе, 
of the Scholars ate at least very disquieting ; especiall 
unteliable ate scholarship awards not made by the univer-y 
sities themselves. | 

There is not the slightest suggestion that extreme cate ir 
not taken in the selection of Scholars, or that the existing 
methods of selection are not used to the best advantag 
But the question is seriously raised as to whether so muc 


one-third of the Scholars do so badly ; and whether ра 
of the money might not better be spent, still within t 
universities, on Zz/erza] scholarships to students who reveal 
theit genuine ability for work at a university level at the 
end of their first or second year there; such internal 
scholarships might be made of greater value as a compensa- 
tion for the students’ gteater expenditure during the fir t, 


year. As a partial alternative there might well be тоғ 
post-graduate scholarships for the pick of the graduates- 
m 





Chapter VIII 
STATE SCHOLARS AND SUBSEQUENT PERFORMANCES 


— WE now come to a mote highly selected group of university 
Scholars. As a general rule the State Scholars, confined as 
they wete, until quite recently, to тоо men and тоо women 
each year, constitute the majority of the best scholarship 
holders in the modern universities at least. They also 
supply a number of Scholars to the ancient universities, the 
average fot the seven years here considered (1920-1926 
awards) going to Oxford and Cambridge being about 50 
men and 20 women per annum. The scholarships wer 
awatded on the results of the Higher School Certificate 
examination, on which also the majority of other modern 
university scholarships are based. So State Scholats ate 
broadly speaking, the top of the list of Scholars in th 
Higher School Certificate examination. 

We will consider first those State Scholars who pto, 
ceeded to modern universities, and compare these шш. 
with those we have discussed in thelast chapter. The tabla! 
on page 121 shows the comparative performance of State 
Scholars (men and women) in modern universities with that 
of all types of Scholars and Non-scholars in provincial 
universities (“modern’ is substituted for ‘provincial, 
because London University is included in the figures fot | 
State scholarships). 

It is evident that State Scholars in modern universities 
on the average do decidedly better than ‘all Scholars.’ 
Especially do they show fewer failures and pass degrees. 
Taking the same criterion as before of a satisfactory result 
for a Scholar (viz. the gaining of at least a second-class 
honours degree), we may say that about 91 per cent. of the 
State Scholars ate satisfactory, against 62 per cent. of © all 
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TABLE XI 
Моревм UNrversrrres 














University All Non- 
State Scholars Scholars ү 
ргорег Scholars Scholars 
No. Percentage Percentage Percentage Percentage 
tst-class honours . | 194 42°5 43°6 27:4 8:5 
2nd-class honours . | 220 48:5 370 346 249 
3td-class honours . 23 570 T4 Ti T9 
Pass degrees . > 15 33 97 256 42:0 
Fal . с . 4 og 2:3 53 16:9 
— (Fail or 
Total . ‚| 456 withdraw) 





The figures (based on the results of 456 State Scholars) are for the years of 
entry 1920-1926, and include results up to December 1929. The universities 
include al! those in England and Wales, except Oxford and Cambridge. 

The results in the last three columns are those given in the previous chapter, 
and so refer to five selected provincial universities. 

It sbould be pointed out that Table XI includes State Scholars at the University 
of London ; and it is found that not quite so high a percentage of State Scholars 
gain first-class honours at London University as they do at other modern universi- 
ties. On the other hand, the total for State Scholars in Table XI also includes two 
universities I* included in the other enquiry on provincial universities and where 
last ‘Classes sem to be slightly more frequently gained than in the five selected 
ptovincial universities; so that on the whole the results piven in this table are 
fairly typical of the State Scholars in the five selected provincial universities dealt 
with in the preceding chapter. It should also be noted that ‘all Scholars’ 
include the stall number of State Scholars at the five selected universities, which 


will slightly raise their average performance. 


Scholars, On the other hand, they do little better than the 
university scholars proper. (But see later the results when 
men and women ate considered separately.) 

We are left with about 9 per cent., or nearly one-tenth of 
this (modern universities) group of State Scholars who prove 
unsatisfactory. Again I do not wish to assert that it was not 
worth while giving a university education to any of the 
third-class honours or pass men who had State Scholar- 
ships. But it seems doubtful whether they were of the 
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intellectual calibre one might expect from those gaining the 
highest scholarships given by the State; or else some 
deficiency of character, failure of health, or other mis- 
fortune, has led to neglect of work at the university. It 
may be teasonably objected that some of the third-class 
honouts students were border-line seconds, and that in 
accordance with what we have said in Chapters I and II 
about ‘luck,’ the variation of ‘form’ or of examiners’ 
standards, then border-line thitds might well have got 
seconds in another test ; but these we may balance against 
the seconds who were bordet-line thirds in the degree 
results, and who on re-testing might well have got 
thirds. 

We find also that this one-tenth of the State Scholats in 
modern universities ate beaten by one-third even of the 
Non-scholars, who gain fitsts ot seconds ; while over half 
of the State Scholars (57 per cent.) who gain second- or third- 
class honours or less are beaten by about 8 per cent. of the 
Non-scholars who gain firsts. This too was befr-> the 
number of State Scholarships was raised frg? 200 pd 
annum to 308, the number awatded in 1931. ( he inctease 
in numbers means that the awatders havé Y. ee much 
farther down the list of order of merit in the examina. 100 for 
scholarships. | | 

A full consideration of the performance of State Scholats 
at Cambridge and Oxford would involve a tefetence to 
some points which will come better in the chapter on 
Scholarships at the Ancient Universities. But I may say 
briefly here that men State Scholars do nearly as well at 
Oxford as all the College Scholats of five of the largest mets. 
colleges massed together, gaining an average class of 1-83 
against ‘all Scholats’ 1-7, the avetage of Exhibitioners 
being the same as that of State Scholars. 
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THe RELATIVE PERFORMANCE OF MEN AND WOMEN 
STATE SCHOLARS 


When we look into the performance of men and women 
we find that the women greatly lower the average perform- 
ance of State Scholars at the universities. At the modern 
universities first-class honours ate gained by 63 per cent. 
of the men State Scholars, but by only 33 per cent. of the 
women ; the percentages of firsts for modern university 
Scholars proper wete: for men 50 per cent., for women 
26 per cent. For each sex, then, the State Scholars are 
better than the university Scholars proper; the approxi- | 
mate equality of these two types of Scholars mentioned 
above is due to the fact that the greater proportion of 
women among the State Scholars lowets their average 
scote. Taking all the universities (ancient and modern), 
the percentages of firsts gained by State Scholars ate: 
men 52:5, women 28:5. Table XII gives the numbers 
and percentages of all classes. 








TABLE ХП 
STATE SCHOLARS IN ALL UNIVERSITIES 
Men Women 
No. Percentage No, Percentage 
15 class . . А З 250 52:5 128 28:5 
2nd class . : . . 183 38:5 256 570 
3rd class . . . А 31 65 51 1г4 
Pass А . К . ІІ 2:3 9 zo 
Fail Р А . А І 02 $ ri 
Total. s Ў 476 449 





N.B.—A number of Scholars who, through death, ill-health, etc., failed to com- 
plete their courses are omitted from the above Table, as ate one man and one 
woman who obtained a Class IV at Oxford. 
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Thus the women not only gain ‘ firsts ’ far less frequently 
than do the men; but they get a greater proportion of 
‘thirds’ and, so fat as our figures go, of failures. Regula- 
tions until 1930 required that the same number of State 
Scholarships must be awarded to women as to men, and 
it is, I think, no secret that the examiners had to go well 
below the marks of the lowest man State Scholar in order 
to include the requisite number of women. In view of 
the poorer performance of the women recorded here, it is 
evident that the change of regulations in 1930 was justified, 
the Scholarships being now awarded to boys or girls in 
propottion to the numbers of each sex taking the Higher 
School Certificate in one or more previous years It 
will be interesting to see how far this regulation brings the 
performances of the two sexes nearer together in their 
final degtee examinations. 

I append a Table showing the relative performances of 
men and women State Scholars in different subjects. 


TABLE XIH 


PERCENTAGE OF Firsts GAINED BY MEN AND WOMEN STATE SCHOLARS 
IN DIFFERENT SUBJECTS 


(The actual number of students taking a given subject is put ini parentheses.) 


Men Women 
Engineering (30) . : « 70% 
Mathematics (89) . д . 59% 
Natural Sciences (162) . - 59% 
Modern Languages (37) . - 57% 
Modern Languages (123) . . 40% 
Classics (57). $ s . 39% . 
e Classics (26) . " a . 38% 
Natural Sciences (33) А . 3695 
History (до). . . - 35% 
Mathematics (41) . `. . 29% 
English (14) . 6 o 21% English (120). |.  .  . 2% 
З History (92) . ; E 17% 


1 See the Report of the Board of Education for 1930, р. 64. Тһе most recent teport 
shows that in 1931 State Scholarships were awarded to 188 men and 120 women 
(List 154, H.M. Stationery Office, January 1932). 
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Two notable facts emerge from Table XIII: 

(1) In every subject the men gain a higher percentage of 
firsts than do the women, except in English, which is only 
taken by 14 men and where the percentages are the same. 

(2) Even in Modern Languages, a subject in which 
women are popularly supposed to excel men, the percentage 
of firsts is higher among the men. 


Chapter LX 


ENTRANCE SCHOLARSHIP AWARDS AT THE ANCIENT UNIVER- 
SITIES: WITH RESULTS FOR OXFORD 


Here, as in the chapter on provincial universities, we 
are only concerned with the award of entrance scholar- 
ships. Internal scholarships are awarded on the results of 
the university work at the end of one or more years ; and 
there seems to be no reason to doubt the general agreement 
that these internal awards prove extremely reliable. 

` The discussion of results at Oxford and Cambridge must 
differ somewhat from that on the modern universities : 

(x) First, we have now not only Scholats but Exhibi- 
tionets to consider, as well as Non-scholats, whose usual 
name, ‘Commoners,’ we shall here adopt. Such Com- 
monets at entrance may later win an internal scholarship ; 
but for our purposes they remain and ate classed as Com- 
moners. 

Exhibitions may be regarded broadly as minor scholar- 
ships—worth now perhaps £30 or £40 against a usual. 
£60, £80, or £100 per annum for scholarships, — ; 

The results for Scholars and Exhibitioners (will be 
shown separately, but for comparison with . provincial 
universities they should be considered together, as pro- 
vincial university Scholars usually include all receiving 
such awards. 

(2) We ignore for the time being pass degtees. Practi- 
cally no Scholars at Oxford and Cambridge take a pass 
degree. We shall compare the performance in the final 
honours degree examinations of Scholars, Exhibitioners, 


1 At Cambridge, for example, internal scholarships are usually given because 
of the gaining of a first-class in Part I of a Tripos ; and it will be seen later that 
this is a very reliable prophecy of success in Part П. 
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and Commoners. "That is, we shall compare the scholars 
with picked Commoners only—those who take an honouts 
degree. As will be seen later, this makes some of our 
results the more striking. 

(3) The question also atises as to whether we ate con- 
cerned with a higher standard of attainment by say first- 
and second-class honouts men at Oxford and Cambridge 
as compared with the modern universities. There is no 
doubt that there are far more men of a very high degree of 
ability at Oxford and Cambridge. The scholarships there 
ate admittedly of higher standard, and the pick of the 
scholars of our best schools will almost invariably choose 
to go to Oxford or Cambridge if they can afford it. 

Admitting, however, that a group of men and women 
at the top of the first-class honouts are, as a group, superior 
to our first-class honouts students at modern universities, 
it does not follow that the lowest standard required for a 
first or second at Oxford or Cambridge is necessarily 
higher than that required for a first or second in the same 
subject at a modern university.! 

1 On this point I have asked the opinion of some half-dozen professots in 
modern English universities who have examined both in modern universities and 
in Oxford and Cambridge. They represent between them six different subjects, 
three Arts and three Science. ‘They all agreed that the minimum requirement for 
a first (or a second) is no higher in an afcient than it is in a modern university. 
Another lecturer consulted said that, as regards Classics, it was difficult to compare 
the standard required for a Tripos Part H at Cambridge with that of the final honours 
at the modern university with which he was familiar; but that some ‘ firsts’ he 
had known at the latter would not have gained a first in Part II at Cambridge. 
Another Cambridge lecturer remarked on tliis, that the converse would sometimes 
be true, that a ‘first’ in Part П might not gain a first in some modern universities, 

An Oxford graduate who is a lecturer in a modern university made enquiries on 
this question among three colleagues who had also been students at Oxford : all 
had examined in a modern university, and one also at Oxford. Together they 
represented among them English, history, and a modern language. АП agreed 
that while among students just reaching a first class the performances on the questions 


set may be as good in the modern university as at Oxford, the questions set at 
Oxford were less strictly confined to a prescribed syllabus, and were often of a 
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For out present purposes, however, it is not a vital 
point to decide the telative standatds of the old and 
modern universities. Weare concerned primarily with the 
telative performance of Scholars and Commoners within 
the same university. 

As to scholatship awards at Oxford and Cambridge the 
following points may be mentioned : 

(а) The competing students are somewhat older than 
those competing for modern university scholatships. In 
many cases a student who sits for a Higher School Cetti- 
ficate in July will compete for Oxford ot Cambridge 
scholarships the following December or spring. This 
greater maturity means greater stability; adolescent 
changes ate more completely over. 

(2) The Scholars are to some extent selected in the 
first instance by the schools themselves, who advise A to 
be content with a modern university scholarship and to go 
to a modern university at 18, but advise B to wait another 
six months and compete for a scholarship at Oxford or 
Cambridge. No doubt sometimes the schools err and, a$ 
a Cambridge tutor suggested to me, fail to advise a boy 
to tty for a scholarship who subsequently proves himself 
well worthy of one. 

(c) A definitely higher standard of attainment is required 
for a scholarship at Oxford or Cambridge than is required 
for ordinary scholarships at а modern university.,/ This 


somewhat different type; so that an equal show of knowledge on the Oxford 
papers required a wider range of reading and a higher degree of ability than the 
teaching of apparently the same standard in a modern university examination, 
Personally, I am inclined to think there is considerable truth in this suggestion, 

especially as regards certain subjects. 

` Indeed, the answer to the whole question varies no doubt considerably, not^ 
merely according to the university concerned, but according to the particular subject 
in a particular university. For as already remarked, the examiners in one subject 
in a university may be generous in giving firsts, while the examiners in another 
subject are grudging, 


" 
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means that the student can be tested on work that more 
neatly resembles the work done at the university itself. 
Also, there are not only papers in the student's special 
subject, but a general or essay paper; though how fat 
the petformance on this general paper is taken into account 
it is difficult to discover. Several Cambridge tutors told 
me it was only brought into consideration when marginal 
cases were being decided ; another that the custom varies 
from college to college. An Oxford teacher, however, 
openly stated at a recent Congress of British universities 
that a colleague of his, in marking the papers for a histoty 
scholarship at Oxford, ignored the special history papers, 
and recommended on the basis of the general paper, with- 
out, he said, any less successful.results than usual! 

The particular types of examination papers, however, 
adopted at Cambridge at least, do not seem to introduce 
any distinction of critical importance in the matter of 
selecting the best men; for I analysed the results gained 
by Scholats at one Cambridge college, where scholarships 
ate usually awarded on the Higher School Certificates and 
not on the special examinations for Cambridge college 
scholarships, and it proved that the Scholars at this college 
did at least as well in their honours degree examinations as 
those of three of the other four large colleges examined. 

Our main problem as to the ancient universities is, how 
do entrance Scholars and Exhibitioners do as compared with 
the best of the Commoners? We will take Oxford first. 


OXFORD UNIVERSITY 


Five of the colleges, including three of the largest, were 
selected. Particulars as to the performance of Scholars 
were supplied through the kindness of the Heads or 
Tutors of three colleges : the results of other colleges were 

9 
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obtained from the University Calendar, in which also the 
tesults sent to me by the colleges wete checked. The 
figures are given below in Table XIV. 


TABLE XIV 


RESULTS IN THE FrNAL Honours ScuHoors For FrvE MEn’s COLLEGES, 
n IN 1925—1930 INCLUSIVE 
Total Number of Students, 1,102 





Commoners taking 


Entrance Scholars Entrance Exhibitioners Honours Degrees 





No. Percentage ? No. Percentage No. Percentage 
Class I . А 91 44:6 30 32 58 72 
Class П . " 81 397 53 56 285 35°5 
Class ШТ. А 27 13:2 їо 10 344 +42°8 
Class ТҮ. y 5 255 2 2 116 14:5 
Totals , А 204 95 803 





Averace Crass 
Scholars, 1:73 ; Exhibitioners, 1-83 ; Commoners, 2°65, 


From Table XIV two things clearly emerge: (1) The 
superiority of Scholars to Commoners in gaining © firsts’ 
(44:6 per cent. against 7-2 pet cent.) is much gteater than it 
was in the provincial universities (27-4 per cent. of all 
Scholars against 8:3 per cent. of all Commoners), although 
at Oxford we are comparing Scholars only with the pith of 
the Commoners, namely, those who take honours degrees. 


2 In the text we have compared ай Scholars with al? Commoners in provincial 
universities, but at Oxford only Scholars (and Exhibitioners), with Commoners 
taking honours degrees; 1 am informed that Scholars practically never take a pass 
degree at Oxford. If it be thought fairer, we may compate also in provincial 
universities only Scholars and Commoners taking honours degrees, We then find 
the following : 

Of Oxford Scholars and Exhibitioners, 40 per cent. (as above) gain firsts. 
Of Oxford Commoners taking honours, 7:2 pet cent. gain firsts, 

Of provincial university Scholars, nearly 4o per cent. gain firsts, 

OF provincial university Commoners taking honours, 20 per cent. gain firsts. 
By this method the relative Superiority of Scholars to Commoners appears still greater 


at Oxford as compared with the superiority of Scholars to Commoners at the 
provincial universities, 
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As, however, * scholarships’ at provincial universities 
include both large and small awards, it is better to compare 
them with scholarships and exhibitions combined at 
Oxford. Adding together then Scholars and Exhibitioners 
at Oxford, we find 40 per cent. of all such gaining firsts— 
still a much larger proportion of firsts than at the provincial 
universities—and about 45 per cent. gaining seconds. 

In comparing the Oxford results, however, with those 
of provincial universities, we must temember that the 
latter included women, who lowered the proportion of 
‘firsts’ gained by Scholars; and also that they included 
many scholarships not awarded by the universities them- 
selves. Confining ourselves to men only and to scholar- 
ships awarded by the provincial university authorities, we 
saw (Table VI, p. 109) that 50 per cent. of men Scholats 
obtained © firsts’ and 31 per cent. ‘seconds.’ Thus these 
provincia] university men Scholars get a higher proportion 
of © firsts ? than do the Oxford Scholars and Exhibitioners 
combined; but the provincial university Scholars get a 
somewhat larger proportion of ‘thirds,’ passes, and 
failures—namely, 19 per cent. against about 15 per cent. 
at Oxford. Taken on the whole, it looks as though the 
two groups of men Scholars do about the same, relatively 
to the standards of their respective universities, but of 
course this is quite consistent with a superior performance 
of the pick of the ‘ firsts’ at Oxford as compared with the 
pick of the ‘ firsts’ in modern, universities. 

(2) The second fact that emerges from Table XIV is 
that the great majority of Scholars and of Exhibitioners 
justify their awards, if we take, as we did in provincial 
universities, the criterion to be the gaining of first- or 
second-class honours. Of Scholars 84 per cent. gain a first 
ot a second, of Exhibitioners 88 per cent. 
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Yet grouping Scholars and Exhibitioners we find that 
about 15 per cent. ате placed in Class ШІ or Class IV, 
which will scatcely be considered a satisfactory pet- 
formance. It is claimed by some Oxford men with whom 
I have conversed that the presence of a fourth-class honours 
at Oxford slightly raises the level of Class. ITI as compated 
with a third at Cambridge, where thete is no fourth class ; 
and it is true-that we find a lower percentage of firsts and 
seconds at Oxford in the final honours schools than we do 
in Part II of the Triposes at Cambridge, which most neatly 
corresponds. 

Suppose we allow this claim, and regard one-third of the 
Oxford third classes as reaching a standard equivalent to a 
low second at Cambridge ot a modern univetsity, then we 
get only about 11 per cent. of Scholars and Exhibitioners 
who fail to jugtify their awards, whereas in the provincial 
universities it was about 38 per cent. of ‘all Scholars’ of 
19 pet cent. of men Scholars selected by the universities. 

Nevertheless, two criticisms can be made of these 
Oxford results : 

(2) The distinction between the awards of scholarships and 
exhibitions seems very insecure. It is true that a higher per- 
centage of Scholars gain firsts (44 per cent. against 32 pet 
cent), but the Exhibitioners show actually a lower percent- 
age of thirds and fourths. Again, 30 Exhibitionersegaining 
firsts beat 113 Scholars getting seconds, thirds, or fourths. 

The average classes 1 gained by Scholars and Exhibitioners 
were almost the same, the figures for the various groups 
beittg : 

Scholars, 1-73 ; Exhibitioners, 1:83; Commoners, 2-65. 

Thus on the average this group of Exhibitioners is almost 


* Calculated by the method used for provincial universities (p 


: - 112), except that 
at Oxford and Cambridge tlie very rare failures and pass degree a 


wards are ignored, 
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as good as the Scholars. Тһе superiority index of Scholars 
to Exhibitioners is only 1-06.1 It should be added, how- 
evet, that the five colleges selected include one in which the 
Exhibitioners actually do as well as the Scholars of the 
other four colleges, and this college provides more than its 
share of Exhibitioners ; but even if this college is left out, 
the remaining Exhibitioners (49 in number) gain an 
average class of 1:97 against an average of 1-81 for the 
Scholars of the same colleges, a superiority index for the 
Scholats of only 1-08. 

Or, looking at it in a different way, in these colleges there 
ate 42 Exhibitioners with firsts or seconds who might have 
-teplaced the 26 Scholars who gained only thirds or fourths. 
Also in the particular college referred to, there wete 21 
Exhibitioners with firsts who might have replaced the 18 
Scholars with seconds, thirds, or fourths. ‘Thus we may 
-still say that the distinction between the awards of scholar- 
ships and exhibitions at Oxford seems insecure. 

It is possible that at Oxford (as I found is the case at 
Cambridge) some colleges find that they have not enough 
scholarships to awatd to men who have been declared by 
the examinets to be of scholarship standard ; and so some 
potential Scholars have to be content with exhibitions. So 
far as this is the case, the grounds for criticism of the 
discriminating power of the examiners ate lessened ; but 
they are not extitely removed, for presumably the exhibi- 
tions would be given to the men who were judged weaker 
than the others; yet many of them beat some of the 
Scholars. Also, even if the examination accuracy avere 
unquestioned, there remains a weakness in the system as a 
whole which labels and rewards as exhibitioner the man 
who proves to be superior to the Scholar. 

1 The superiority index is explained in Chapter VII, p. 113. 
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(b) Perhaps mote serious is the fact that 58 Commoners 
gaining first-class honours beat 113 Scholars and 65 Ex- 
hibitioners who got seconds, thirds, and fourths ; and 285 
Commonets gaining Class II beat 32 Scholars and 12 
Exhibitioners who got thirds and fourths. Even ignoring 
the group of Commoners who got seconds, we may infer 
that a sounder award would have given scholarships or 
exhibitions to the 58 first-class Commoners instead of to 
the 44 Scholars and Exhibitioners who got only thirds or 
fourths, and to 14 of the weakest of those who got seconds. 

This gives us 58 Commoners who got no award, but who 
deserved one, out of the 299 awards made. ‘That is, about 
19 per cent., ot roughly one-fifth of the awards were misplaced ; 
not in the sense that all these Scholars did so badly as to be 
undeserving of a scholarship or exhibition, for most ob- 
tained at least a second class, but in the sense that there 
were other better men who obtained no award. 

No doubt it may be replied that the classing in the 
Honours Finals is not infallible. There is of course no 
absolute standard ; and no doubt some of the best Scholars 
in Class II might, with other papers, or on a different day, 
ot with different examiners, have obtained a fitst ; so, how- 
ever, might some of the Class II Commoners ; and con- 
versely some of the Class,I Scholars might have obtained a 
second. Uncertainties then seem to balance one another. 

We might indeed go further and say that the 30 Ex- 
hibitioners who got firsts as well as the 58 Commoners 
(in all 88) were more deserving of a scholarship than the 32 
Scholats who got thirds or fourths and the weaker of the 
Scholars who got seconds. This would give us 88 mis- 
placed awards out of 204 scholarships, that is, 43 per cent., 
ot about two-fifths, the same proportion as we found by 
a somewhat similar method in the modern universities. 
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Leaving on one side, however, the distinction between 
Scholars and Exhibitioners and confining ourselves to Com- 
moners with firsts, we may at least say that about one-fifth 
of the awards were misplaced, judging by the performance 
of students in the final degree examinations. 

As with the provincial universities, we have to consider 
whether the possession of adequate means rules out some 
men from scholarships who would otherwise get them. 
From my enquiries, however, I gather that at Oxford and 
Cambridge such men would almost invariably be enrolled 
as Scholats, even if they did not receive any emolument. 
There remain, however, a small number who, not needing 
financial aid, refrain from competing for a scholarship, and 
among these there may be some who would gain firsts and 
so improve the records of the Commoners. It will, how- 
evet, I think be agreed by those familiar with conditions at 
Oxford and Cambridge that such students are not numerous 
enough to account for the above facts to any substantial 
degree. 

As already suggested, the schools no doubt make oc- 
casional errors in recommending some pupils not to try fot 
scholarships, who afterwards beat Scholars in the degree 
examinations. So far as this is the case, there is a weakness 
in the general system of selecting, Scholars ; and incident- 
ally a further argument in favour of the substitution of 
additional internal scholarships or exhibitions for some of 
the entrance exhibitions—as will be suggested later. 

Finally, we have to bear in mind that at Oxford (as at 
Cambridge) some of these Commoners who do so well in 
the final honours examinations will have been awarded 
internal scholarships on the earlier examinations * Modera- 
tions? (at Cambridge, Part I of a Tripos or the * Mays’), 
so that errors in the awards of entrance scholarships at the 
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ancient universities ate to some extent compensated for 
latet, which is not so to anything like the same extent at 
most modern universities. 


WOMEN STUDENTS AT OXFORD 


Similar analyses were made of the degree results of the 
students of two women's colleges at Oxford over a petiod of 
six years (1925-1930) for one college and seven (1922- 
1928) for the other. The results are given in Table XV. 


TABLE XV 
Women STUDENTS AT Oxrorp. ResuLTS oF Ема, Honours EXAMINATIONS 
(Total number of students, 389) 





Scholars Exhibitioners Commoners 
No. Percentage No. Percentage No. Percentage 
1st class . Я 14 22 8 23 15 5 
2nd class 35 54 24 71 163 56 
3rd class . А 12 19 2 6 100 24 
4th class . Я 3 5 - - 15 5 
Totas .| 64 34 ‘291 





The numbers here are small for the purpose of analysis. 
But so far as they go, they indicate the following : 

(т) Most striking fact of all, perhaps, the Exhibitioners 
actually do better than the Scholars. The average class 
gained by each group is as follows : 

Scholars, 2:06 ; Exhibitionets, 1:82; Commonets, 2:58. 

This performance of the exhibitioners is not due to the 
special excellence of the exhibitionets in one particular 
college, as was partly the case with the men. 

It is possible that for these colleges the number of 
candidates of undoubted scholarship standard is so great 
that exhibitions have to be awarded to women who ate 
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declared by the examiners to be practically as good as those 
to whom Scholarships are awarded. If so, we should say 
that the error of selection lies less with the entrance exam- 
ination, and rather with the whole system that gives stu- 
dents of practically equal capacity an inferior label and a 
smaller award. . 

(2) Grouping Scholars and Exhibitioners together, we 
find that 23 per cent. get firsts, against 4o per cent. of the 
men. The proportion, however, of the women who get 
lower than a second (17 per cent.) is about the same as that 
of the men (15 per cent.); so that if we consider what 
proportion of the women fail to justify the award of the 
scholarship ot exhibition, we find it is about the same as 
that of the men—a result very similar to that found in the 
provincial universities. 

If, as with the men, we allow one-third of the third classes 
to reckon as equal to low seconds at Cambridge or a 
provincial university, then 12 per cent. of the women 
Scholars and Exhibitioners fail to justify their awards— 
against 11 per cent. of the men. (In the provincial univer- 
sities the figures were: men, 19 per cent.; women, 18 
per cent.—for Scholars selected by the universities them- 
selves.) 

A sounder award would have given scholarships at 
least to the 15 Commoners who gained firsts instead of 
to the 15 Scholars who gained only thirds or fourths ; so 
that at least 15 scholarships out of 64 went astray in this 
sense—about one-quarter, against one-fifth among the 
men (by the same method of reckoning). 

Or following the stricter method, we may say that the 
8 exhibitioners as well as the 15 Commonets who got 
firsts were more deserving of a scholarship than the 15 
Scholars who got thirds or fourths, and the weaker of the 
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Scholars who got seconds. This would give us 23 mis- 
placed awards out of 64 scholarships, or one-third—nearly 
as latge a proportion as among the men. 

As with the men, howevet, we must make some allow- 
ances for women Commonets who gain firsts, but who 
never competed.: for scholarships. 

(3) The performance of the women Commoners is on 
the whole better than that of men Commoners taking 
honours schools. A slightly lower percentage of women 
gain firsts, but a lower percentage gain only thirds or fourths. 
The average class of the women Commoners is 2:38, 
slightly better than the 2:65 for the men, not surprising in 
view of the keen competition for entrance to the women's 
colleges at Oxford—as at Cambridge. 

As the women Scholars (and Exhibitioners) do worse 
than the men Scholars (and Exhibitioners) it follows а 
fortiori that the superiority of Scholars to Commoners among the 
women is not so great as it was among the men. 

(4) There is a striking resemblance between the women 
Scholars (and Exhibitioners) at Oxford and the women 
Scholars of the provincial universities, in the percentage 
who gain firsts, seconds, and lower than seconds. As, 
however, the percentage of firsts is not so high among the 
men Scholars'at Oxford,as in the provincial universities, 
we may say that the superiority of men to women Scholars in 
Oxford is not so great as it is in the provincial universities (con- 
fining ourselves to scholarships awarded by the provincial 
universities themselves). This, together with the relatively 
good performance of women Commonets at Oxford, 
confirms the general impression that even relatively to the 
men, the competition for scholarships (or even admission) 


for women at Oxford is greater than it is at the provincial 
universities. 


Chapter X 


ENTRANCE SCHOLARSHIP AWARDS AT CAMBRIDGE, AND 
A COMPARISON WITH RESULTS AT OXFORD 


'Turons at three Cambridge colleges very kindly provided 
me with figutes showing the performance of Scholars and 
Exhibitioners in their own colleges. These were checked 
by reference to the University Calendar, from which were 
extracted also the performances of Commoners in these 
colleges, and of all students in two other colleges. Thus, 
figures from five of the largest men’s colleges were ob- 
tained for the years 1922-1929, providing a total of 2,904 
students taking honours degrees. 

At Cambridge the degree examinations are somewhat 
mote complicated than at Oxford, from our present point 
of view. Usually an honours student takes two * ітіроѕ? 
examinations, first in Part I at the end of one or two years, 
and then a mote advanced examination (Part IT) a year or 
two later. The examination in Part ЇЇ is usually the closest 
parallel to the Oxford Final Honours examinations. But 
there ate two modifications to this general rule : 

(а) Some students, instead or taking Part П in the same 
subject as that in which they took the Part I examinations, 
change over to another subject, and take the Part 1 or even 
Part II examination in the new subject. Sepatate figures 
will be given for such students. 

(E) Some students are content to take Part I only. A 
degtee can be obtained on Part I only in certain subjects, 
fot example, the natural sciences; and some triposes, 
e.g. the mechanical sciences tripos, ate not divided into 
Part I and Part П. 

Part I of the tripos examinations is usually taken at the 
end of the second year, and sometimes even at the end of 

139 
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the first year especially by Scholars and exhibitioners in 
mathematics and classics. 

In order best to compare the performance of Scholars, 
Exhibitionets, and Commoners in Part I examinations, we 
will first exclude those subjects (like law, moral sciences, 
anthropology, theology, mechanical sciences, etc.) which 
are not the subjects in which the scholarship examinations 
are taken ; and also English, in which there was no separa- 
tion into Part I and Part П until 1929. This still leaves us 
with the triposes which attract by far the larger proportion 
of honours students. 


The results are given in Table XVI. 


TABLE XVI 


RzsuLTS iN Tros EXAMINATIONS, Part I, For Five Men’s CoLLEGES, 
1922-1929. SUBJECTS : MATHEMATICS, CLASSICS, MODERN LANGUAGES, HISTORY, 
NATURAL SCIENCES 

















Entrance Scholars Entrance Exhibitioners Commoners 
No. Percentage No. Percentage No. Percentage 
1st-class hons. . 510 76:2 115 4171 123 то'о 
2nd-class hons. . 82 201 140 500 562 45°5 
3rd-class hons. . 15 3:7 25 8:9 549 44:5 
"Totals . . 407 280 1,234 





In this first hurdle after entry to the university we see 
that nearly all Scholars and Exhibitioners justify their 
awatds, if we take again as our criterion the gaining of 
frst- or second-class honours. Only 3-7 per cent. of 
Scholars and 8-9 per cent. of Exhibitioners get lower than 
a second. 

It is true that 123 Commonets gaining firsts beat 97 
Scholars and 165 Exhibitioners who only get a second or 
a third. But this comparison cannot at this stage fairly 
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be made, as we do not know how many Commoners took 
two or even three years to Part I against one or two years 
taken by some Scholars. 


RESULTS OF Part П Твіроѕ EXAMINATIONS 


As we have said, Part II examinations are mote com- 
parable with the final honours at Oxford in most subjects. 
Let us consider now Part II results for the same colleges 
and the corresponding period (one year later) which we 
have studied for Part I. 

Of the students considered under Part I, the following 
percentages passed in Part II one or two years after Part I: 

Scholars, 72% ; Exhibitioners, 53% ; Commoners, 38%. 


(Part П is taken in the same subject as that taken in Part I 
by the great majority of these students, approximately 85 per 
cent. of Scholars, 81 per cent. of Exhibitioners, and 75 pet 
cent. of Commoners. 1) 

It will be noticed that at this stage we have a further 
process of selection: only a certain percentage of Com- 
mohets take a Part I tripos, and of these only 38 per cent. 
go on to Part П; so that to an even greater degree than in 
Part I, we ate comparing Scholars with picked Commonets. 
Table XVII, p. 142, shows the results of all these in Part II 
examinations. 

In this group 92 per cent. of the Scholars and 81 pet 
cent. of the Exhibitioners get a first or second, and so we 
may say that, taken together; 88 per cent. justify their 
awatds, if we ate satisfied with that criterion. (The figure 
is 9o per cent. if we confine ourselves to Part II examina- 


1 The term ‘same subject’ is used here in a broad sense. In some triposes, 
Part П would include examinations in material considerably different from that 
studied for Part I—as in Classics, where Part П may deal largely with Ancient 
History or Philosophy. In the Natural Sciences tripos the student can take one 
subject instead of three or four taken for Part I. 
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TABLE XVII 


Part H oF ALL Trrposes, NUMBER AND PERCENTAGE GAINING Firsts, 
SECONDS, ETC. 


* 








Scholars Exhibitioners Coramoners 
No. Percentage No. Percentage No. Percentage 
rst class . А 157 557 59 347 80 II'I 
and class , : 102 36:2 78 45°9 333 46:0 
3td class . . 25 81 35 19'4 310 429 
282 170 723 
Average Class . 1:52 1:84 2:32 





In this table we give results for Part II o£ a// triposes, as these are more com- 
parable with the Oxford figures, which include some students who take at the 
university subjects other than that in which they won their scholarship, and other 
students who take * moderations ’ (the earlier examination) in their scholarship 
subject—say classics, and who then change over to another subject, say history. 
Later we shall give another table of Part II results, confining ourselves to those 
students who took Part II in the same subject as that in which they took Part I, 
and it will be seen that the various percentages of successes are very similar to 
those given above in Table XVII. \ 


tions taken in {һе saze subject as that taken for Part І.) 
I may say in support of this criterion at Cambridge that the 
senior tutor of one of the largest colleges told me they 
had adopted the following attainments as tests of satis- 
factoty performance by their Scholars. They thought a 
Major Scholar ought to get a first in Part I and in Part. II 
of the triposes; a Minor Scholar should get at least a first 
in Part I and a second in Part II, and an Exhibitioner at 
least a second in Part I and a second in Part II. 

Accepting, then, the criterion of “at least a second 
class,” we see that only about 12 per cent. fail thus to 
justify their scholarship or exhibition award. (The figure 
was 15 per cent. for Oxford or 11 per cent. if we allowed 
one-third of the Oxford third classes to reckon as equivalent 
to a Cambridge low second class.) 
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Combining Scholars and Exhibitioners in the above 
table, we find that nearly 48 per cent. gain firsts, as with 
Oxford a great superiority over the Scholars (men and 
women) of provincial universities, where only 27 pet cent. 
gained firsts, but almost the same proportion as among 
men Scholars only, selected by the provincial universities 
themselves (5o per cent.). Table XVIII gives a convenient 
summary for the purpose of this comparison. 


TABLE XVIII 


COMPARISON OF MEN SCHOLARS (AND EXHIBITIONERS) AT OXFORD, CAMBRIDGE, 
AND PROVINCIAL UNIVERSITIES (RESULTS OF FINAL EXAMINATIONS) 





Provincial Universities (Scholars 








Cambridge (Tripos, 3 na 
Oxford V Put ft Ab eta Y oi 

Percentage! Percentage 
īst class. ‚| 405 | rst class ‚| 478 |1stclss . |500 
2nd class . . | 448 | 2nd class К 39:8 | 2ndclass . |3ro 
зга class . . | 124 | 3rd class . | 124 | 3rdclass .| 48 - 
4th class . . 2:3 Pass. ‚| gts 32 

Fail, or 


withdrawn | $'1 





Accuracy OF DISCRIMINATION BETWEEN SCHOLARS, 
EXHIBITIONERS, AND COMMONERS 


In one respect the award of scholarships seems more 
reliable at Cambridge than at Oxford ; for the superiority of 
Scholars to Excbibitioners is mote decided than it was at 
Oxford, as will be seen from the average class gained and 
from the fact that not only the Scholars’ percentage of 
firsts is much higher than that of the Exhibitioners, but 
their percentage of thirds is lower, as was not the case at 
Oxford. 

Nevertheless, in Part II of the triposes, 80 Commoners 
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getting firsts beat 125 Scholars! and 111 Exhibitioners 
getting seconds or thirds. A more accurate selection 
would have given awards to these 80 Commoners at least, 
instead of to the 56 Scholars and Exhibitioners who got 
only thirds, and to the 24 weakest who got relatively low 
seconds. On this estimate 80 awatds out of 452 went 
astray, that is 18 per cent., remarkably close to the 19 per 
cent. which a similar way of reckoning gave us in the 
Oxford awards (see p. 134). (If we confine ourselves to 
Part П examinations, in which the same subject was taken as 
in Part I, this percentage for Cambridge becomes 18:7.) 
We might indeed go further, and say that errors were 
made in not awarding scholarships to those Exhibitioners 
who gained firsts in Part IT, and so proved more worthy of 
scholatships than did the Scholars who got only a second 
or a third. Along these lines we should get the follow- 
ing estimate: all the 125 awards to Scholars who get 
seconds (102) or thitds (23) might better have been given 
to a like number of Exhibitioners and Commoners who 
get firsts (139). By this estimate the total errors in awards 
of scholarships (only) would be 125/282, that is 44 pet 
cent.—a vety similar figure to that for Oxford calculated 


1 This superiority of Scholars to Exhibitioners at Cambridge would be greater 
still were it not for the fact that the Exhibitioners of one large college (A) do about 
as well as the Scholars of the other colleges, and nearly as well as the Scholars of 
their own college. Thus 59 per cent. of the Exhibitionets of this college get firsts 
in Part II against 63 per cent. of the Scholars. The question arose as-to whether 
the excellent performance of the Exhibitionets at College A was due to a larger 
proportion taking classics or mathematics (in which triposes firsts are more fre- 
quently gained), Analysis, however, showed that this was not so. Of the 
Scholars at A a considerably larger proportion take classics or mathematics than 
is the case at other colleges. In Part II of other triposes the Scholars of College A 
do vety little better than those of other colleges, and not quite so well as their own 
Exhibitioners 1 The number of Exhibitioners, however, in this group is only 26. 


Details of the average classes obtained in the various colleges for Part I are given 
in Appendix П (с), р. 190. 
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in the same way, which was 43 per cent. (see p. 134). 
This, however, is no doubt a severe criterion of reliability. 
If we overlook the award of mere exhibitions to men who 
finally beat many of the Scholars, on our previous method 
of teckoning we find that about one-fifth of all these 452 
awards for the period in the five colleges have gone astray, and 
Should have gone to Commoners. 

How far this misplacement is through the erroneous 
estimation of ability in the entrance examinations we can- 
not say. Three supplementary causes of the deterioration 
of the Scholars have been mentioned to me by Cambridge 
tutors. One is that some Scholars with first-class ability 
have not the qualities of character needed to ensure steady 
work in the greater freedom of university life as compared 
with school. A second is that a few break down in health. 
A third is that some Scholars come up to the university 
coached and crammed to their utmost capacity ; and someof 
their special abilities (perhaps chiefly for absorption) could 
be made most of only under close supervision and in- 
struction. In the different and more individual work for 
advanced honouts courses they are found lacking, and ate 
passed by others with more of the special capacities 
needed, but who wete not so thoroughly prepared for the 
enttance examination. One tutor told me that they 
recognised that School x (a big public school) sent its 
pupils up thoroughly prepared for the scholarship examina- 

tions, but that pupils from School y, though they did 
worse in the scholarship examination, usually surpassed 
the former in higher university work. This of course is a 
rather vague and general impression and, like all such, 


1 Tt is true that, as pointed out in footnote 1 on p. 144, the exhibitioners at 
one particular college did exceptionally well. Yet considering the awards within 
this college alone, those ‘ gone astray’ were almost two-fifths (following the 
same method of reckoning). 

10 
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needs confirmation; and it suggests a fruitful line of 
enquity in both the ancient universities as to the telative 
performance from different leading schools in (a) entrance 
scholarship examinations, and (P) success in Part I and 
especially Part II triposes. | 

If it be replied that this would be an objectionable 
proceeding, it should be pointed out that the general 
imptession already exists in some minds, and it is desirable 
at least that such should not exist on a false basis. 


STUDENTS TAKING Part I or Two DIFFERENT TRIPOSES 


Let us consider now those students who, having taken a 
Part I ttipos, proceed to take another Part I ttipos in a 
different subject, instead of taking a Part II examination. 

Judging from a petiod of six years, over half the Scholars 
and Exhibitioners who do not take a Part II tripos take a 
second Part I in a different subject. Some of the remaining 
students will be medical students only taking Part I in 
natural science and taking their degrees on that part; 
others will take the mechanical sciences tripos (on which 
also a degree can be taken), both of which may be vety 
creditable performances for Scholars. 

As to Scholats and Exhibitioners taking a second Part 
I, Table XIX, p. 147, shows the diffetent classes gained 
by this group of students. 

The most notable fact here is that Exbibitioners do practi- 
cally as well as Scholars in the second Part I tripos which 
they select, though they were definitely infetior as a group 
to the Scholars in the first Part I, which would be as a rule 
in the subject in which they won their scholarships. 

Combining the Scholars and Exhibitioners again, we 
find that in this second Part I 88 per cent. gain firsts or 
Seconds, exactly the same proportion as those who gain 
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TABLE XIX 


SHOWING CLASSES GAINED IN BOTH Triposes BY STUDENTS TAKING 
Parr I гч Two DIFFERENT SUBJECTS 










Scholars Exhibitioners 








In the first | In the second 
” Раг 1 Part I 






In the first |In the second| 
Part I Part I 

















ClassI . 24 Class I . > 10 15 
Class П . 10 16 Class II ; 18 14 
Class Ш I 3 Class Ш . 4 5 





35 





32 32 








firsts or seconds in a Part П. So this smaller group also 
justify their awards to the same extent, if we accept as a 
ctitetion the gaining of a second class even in Part I of a 
second ttipos, on the ground that some credit must be 
given for doing so satisfactorily in a new subject. 

. As the number .of Scholars and Exhibitioners taking 
Part I of two triposes was small, and as the labour of 
tracing in the Calendar all the Commoners doing the same 
would have been vety great, a compatison with Commonets 
was not made. So we cannot say what proportion of these 
scholarship and exhibition awards were misplaced in the 
sense used before. 


SHouLD FEWER ENTRANCE SCHOLARSHIPS (OR Exurst- 
TIONS) AND MORE INTERNAL SCHOLARSHIPS BE AWARDED ? 


‚А question of some practical importance arises as to 
whether it would be better to be stricter in the award of 
entrance scholarships (possibly by awarding no exhibitions), 
and to awatd more internal scholarships on the results 
of Part I examinations. (A recommendation of this kind 
was made for provincial universities in Chapter VII.) 

The proposal raises the difficult problem as to whether 
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many students of the type who now gain entrance 
scholarships would be able to affotd even one year at 
Cambridge without a scholarship. But leaving this 
difficulty for the moment, let us consider whether the 
results of Part I tripos examinations would be a mote. 
reliable basis for the award of scholarships than ate the 
present examinations for entrance scholarships. We take 
again performance in Part II as our criterion of success. 

First let us consider those students (the great majority) 
who take Part П, if at а],. the same subject as that taken 
for Part I (or in a selected subject of the same group). 

Taking our earlier ctiterion—the gaining of a first of 
second in Part П as justifying the award of a scholarship— 
we find as follows, for a petiod of six yeats : 

Of all Scholars, 13 out of 257, Of 5*1 pet cent. get only a 
Class Ш in Part Пл 

Of all students gaining a first in Part I, 14 out of 387, of 
3°6 per cent., get only a Class III in Part II. 

So, from this point of view, there is little to choose 
between (а) Scholars, and (/) a// students (whether Com- 
mohners, Exhibitioners, or Scholars) who gained a first in 
Part I. 


Let us now use the mote exact means of comparison, 
viz. the average class gained. 


Average class gained in Part II by 257 entrance Scholars — 
143. 

Average class gained in Pårt II by 387 students who had 
gained a first in Part I = 1-41. 


Here again we find that a first in Part I is no better, 
augury for success in Part П zz the same subject than is the 


1 This percentage differs from the 8 per cent. in Table XVII, p. 142, since the 
latter includes Scholars who took a different subject in Part П from Part I. 
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gaining of an entrance scholarship at least a yeat, ot more 
frequently eighteen months, before the taking of Part I. 
This is a valuable testimony to the entrance scholarship 
examination. 

Exhibitions. —So fat, however, we have only considered 
Scholars. If we take Exhibitionets, we find that of 149 
Exhibitioners proceeding to Part II in the same tripos as in 
Part I only 57:6 per cent. get a first, while 16-1 per cent. get 
only a third. So Exhibitioners, as a group, are definitely 
inferior to the whole group of men who gained a first in 
Part I. This latter group, however, includes nearly all the 
Scholats. Suppose we consider the proposal to award 
entrance scholarships as hitherto, but to reserve the money 
now spent on entrance exhibitions for the award of scholar- 
ships on the results of Part I tripos examinations ; would 
such awatds be mote reliable ? 

To decide this we must assume that the Scholars have 
already been selected at entrance. Hence we must compare 
Exhibitioners as a group with the group of Non-scholars 
who have gained a first in Part I. This gives us the follow- 
ing figures : 


Of Exhibitioners, 56 out of 149 or 37:6 per cent. gain а 
first in Part IT, and 16-1 per cent. get a third. 

Of all students (other than Scholars) gaining a fitst in 
Part I, 91 out of 163, or 56 per cent., gain firsts in 
Patt П, and only 9 out of 165, or 5-5 per cent., get a 
third. 


It is evident that, leaving Scholars out of consideration, a 
first in Part I is a much better augury of success in Part II 
than is the winning of an entrance exhibition. "Thus # 
would be more reliable to award exhibitions as internal scholarships 
on the basis of the gaining of a first in Part I. 
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On the other hand, those colleges which ate already able 
to give a scholarship to every Commoner who gains a first 
in Part I would not be well advised to reserve any exhibi- 
tions for those who gain a second in Part I. For of those 
who gain a second in Part I (even including Scholars) only, 
44 out of 347 (ot 12 per cent.) gain a first in Part П, while 
25 pet cent. get only a third, a definitely worse petformance 
than that of entrance Exhibitioners. 

From the enquiries I have been able to make, I gather, 
however, that not all colleges can give scholarships to all 
students who gain a first in Part I; while one of the largest 
colleges pays considerable attention to personal qualities of 
students and the reports of college tutors in awarding intet- 
nal scholarships, so that a man who gains a first in Part I 
may fail to win a scholarship, while another with a very good 
second may be awatded a scholarship. I may add here I 
learnt that in the award of enttance exhibitions some col- 
leges (if not all) give great weight to reports on personal 
qualities of the individual applicants. The less one regards 
examination awards as infallible, the more one will welcome 
the influence which such other qualities might be allowed 
in the winning of internal scholarships. 

Let us now consider the performance in a tripos Part П 
taken in a different subject from that taken in Part 1. 
Here our numbets are mote restricted. They suggest 
decidedly, however, that transference-to a new subject for 
Part II is made mote frequently by those Scholars whose 
results in Part I are disappointing. Thus, among Scholars 
who transfer to a new subject for Part П, 18 out of 52 had 
failed to get a first in Part I, against only 33 out of 257 of 
ce на who persisted with the same subject for 

art П. 


When divided into classes (as in Appendix П (4), p. 190) 
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the number in each subgroup is very small fot a comparison 
of percentages. It is safer therefore to consider the average 
class gained in the new Part П. 


These ate as below: 


TABLE XX 
PERFORMANCE IN TrIPos Part П TAKEN IN А New SuBJecT 
Average class of all Scholars — . 1:92, based on 52 cases. 
Average class of all students who une a first i in Part I 1:76, based on 54 cases. 
Average class of all ExJibitioners x 2:05, based on 36 cases. 
Average class of all students (other than Scholars) sio 
gained a first in Part I . . s . Р - 1°65, based on 20 cases. 


It will be seen that for this small group a first in Part I 
was a better auguty for success in Part II than was the 
gaining even of an entrance scholarship. 

As regards Exhibitionets, these gain on the average 
something worse than a second in Part II (2-05), whereas 
the class gained in Part П by all students (other than 
Scholars) who gained a first in Part I was 1-65. 

So that again the award of the exhibitions would have 
been more accurate on the basis of Part I; indeed, for this 
small group the same is true for scholatships. 

Summing up this section, we may say that the award of 
exhibitions at least would be decidedly safer if it were 
deferred until Part I was takenalways taking success in 
Part II as our criterion. 

In many colleges, however, it may be possible to give 
scholarships to all who gain a first in Patt I (or a first class in 
fitst-yeat ‘ Mays’ examination), as well as to move up to 
scholarship rank those entrance Exhibitioners who gain a 
first in Part 1; and apart from this, no doubt serious 
economic objections may be raised against the deferring of 
awards to men who only teach exhibition standard in the 
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admission examination; though against these factors can 
be set the greater possibility of considering personal quali- 
ties in the award of internal scholarships. Such balancing 
of pros and cons, however, is not our present concern, 
which is merely with reliability as tested by success in the, 
highest examination. 


WOMEN STUDENTS AT CAMBRIDGE 


I am indebted to Miss Major, late Mistress of Girton 
College, to het successor, Dr. Wodehouse, and to Miss 
A. B. Dale, of Newnham College, for figures from their 
respective colleges, the one for six years (1926-1931), the 
other for seven yeats (1924-1930). Analysis gave the 
following results : 











TABLE XXI 
Women STUDENTS ar CAMBRIDGE TRIPOS EXAMINATION, Parr II 
Scholars Exhibitioners Commoners 
No. Percentage No. Percentage No. Percentage 
ist class . . 33 25:7 10 154 34 91 
2nd class . . РИ 68-4 43 66-2 251 673 
3rd class . % 11 T9 12 18:4 ' 88 23:6 
Totals . . | 139 100°0 65 100°0 373 I100'0 
50 








As with Oxford, the numbers of women ate small for 
analysis, but we may note the following : 

(1) The discrimination between Scholars and Exhibi- 
tioners is much better here than in the two Oxford 
colleges. The Scholars have more firsts than the Exhibi- 
tionets in proportion, and dicidediy fewer thirds. The 
average class gained by each group was : 

Scholars, 1-84; Exhibitioners, 2:03 ; Commoners, 2:14. 
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(2) Of the Scholars and Exhibitioners, about one-ninth 
fail to obtain second-class honours, and so do not justify 
their awards; 11.3 per cent., to be exact, against 12:4 per 
cent. of the men—the corresponding figures at Oxford 
being 17 pet cent. for women and 14:7 per cent. for men. 

(3) Grouping Scholars and Exhibitioners together, we 
find that 21 per cent. get firsts, against about 48 per cent. of 
the men (of the Oxford women, 23 per cent. got firsts 
against 40 per cent. of the men). So the best women 
(among the Scholars and Exhibitioners) at Cambridge do 
not as a group compare quite so favourably with the best 
men as is the case at Oxford. Asa prejudiced Cambridge 
man, I leave it to the reader to judge whether this result is 
due to the Cambridge women being inferior to the Oxford 
women, or whether it is because the Cambridge men ate 
superior to the Oxford men! 


Тне RELATIVE RELIABILITY OF SCHOLARSHIP AWARDS IN 
DIFFERENT SUBJECTS 


Ate the awards of scholarships in, say, mathematics or 
classics, mote reliable than those for history or modern 
languages i ? This can best be studied by’ comparing the 
average class gained in each subject by the Scholars with 
that of the Commonets—as was done with the provincial 
universities. The separation of the tripos results into 
different subjects leaves us with too few numbers in some 
ttiposes on which to base any teliable conclusions, but the 
results for the six largest triposes (Part I) are given on p. 154. 

It will be seen that the best average fot Scholars is in 
classics ; itis nearly a first. Indeed, 99 out of 116 Scholars 
got a fitst in Part I. Very close is mathematics, while the 
lowest average class for Scholars is in modern languages. 
It must be borne in mind, however, that Part I in classics 
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TABLE XXII 


AVERAGE CLASS GAINED BY SCHOLARS, EXHIBITIONERS, AND COMMONERS IN 
DIFFERENT SUBJECTS OF THE Тк1роѕ EXAMINATIONS, Parr I. (The numbers 
in parentheses give the number of students in the category.) 





Mathematics Classics Natural Sciences 























Average Class Average Class Average Class 

Scholars . А ` r'21 (126) rr$ (116) 1'39 - (83) 
Exhibitioners р n r46 (69) r6; (43) | r8 (74) 
Commoners ; А 2:34 (143) z16 (93) 2:32 (434) 

History Modern Languages | Mechanical Sciences 

Average Class Average Class Average Class 

Scholars. . А 31 (32) r52 (50) r34 (29) 
Exhibitioners А s r82 (45) rós (49) r47 (17) 
Commoners И н 2:44 (350) | 2:32 (214) 2:46 (267) 





and mathematics and history are admittedly preliminary ex- 
aminations, while mechanical science is a final examination, 
and natural science Part I may be. Further, the simple 
figure cannot be taken alone. The examiners for one 
tripos may be more generous than those fot another. To 
get the relative reliability of the scholatship awards in 
different subjects, we must first compare the performance 
of Scholars and Commoners in the same tripos. Then the 
tatio of the average class of the Commoners to the average 
class of the Scholars gives us the index of superiority of 
Scholars to Commoners in that subject; and similarly for 
Commonets and Exhibitionets. We arrange these supeti- 
otity indices in order of size in Table XXIII. 


1 A table is given in Appendix П (D), р. 189, showing the frequency with which 
firsts, seconds, or thirds are awarded in the various triposes. 
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TABLE XXII 
SUPERIORITY InpicEs N Trreos, Parr I, OF 
Scholars to Commoners Exhibitioners to Commoners 
т. Mathematics . , - 193 т. [Mechanical Sciences. .or67! 
2, Classics . . . + r88 2, Mathematics . . . r6o 
з. History . . - 186 3, Modern Languages . . 1°40 
4. Mechanical Sclences . - r83 4. History . : . . 1°34 
5. Natural Sciences ; . 67 5. Classics . л . 1°29 
6. Modern Languages. . 153 6. Natural Sciences . . 1528 


1 Only 17 Exhibitioners in mechanical sciences. 


As to Scholars, it will be seen that mathematics and 
classics draw most reliably the distinction between Scholars 
and Commoners, not unexpectedly perhaps. Probably the 
papers set in Part I of the tripos in these subjects resemble 
the scholarship papers mote than is the case with other 
triposes, though the high degree of discriminative powert is 
not shown by classics when distinguishing Exhibitionets 
from Commoners. It may be surprising to find history so 
vety close to classics and mathematics in the table. Ihave 
heatd in Cambridge the idea expressed that scholarship 
awards in history and English were especially unreliable. 
Yet the superiority of Scholats to Commoners in history is 
neatly as great as it is in classics. Тһе number of Scholars 
in history in this group was small, however, viz. 32. It 
must also be remembered that if a large number of very 
weak students select a given subject, they will lower the 
average petformance of the Commoners, and so raise the 
superiority index. 

Delicacy of discrimination at the top is probably best 
shown by the superiority of Scholars to Exhibitioners in a 
given subject. ‘This is as follows : 

SUPERIORITY INDEX oF SCHOLARS TO ExHrBITIONERS (TRIPOS, PART J) 


т. Classics . * А . r45 4. Mathematics . : . DI 
2. History . . . r39 5. Mechanical Sciences . . TIO 
3. Natural Siento ^ . 130 6. Modem Languages . ‚год 
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Again, it is surprising to find this discrimination between 
Scholars and Exhibitioners more accurate in history than in 
mathematics. Of coutse, if in a given subject there is a 
latge numbet of men of high and approximately equal 
ability, a number sufficient to absorb not only all scholar- 
ships but all exhibitions also, then this would lower the 
superiority of Scholars to Exhibitioners. It is possible that 
this may occur to a greater extent in mathematics and 
classics than in history. 

If we average all the three supetiotity indices (Scholars 
to Commoners, Exhibitioners to Commonets, and Scholars 
to Exhibitioners) we find mathematics most reliable—by 
a vety small margin, classics and history being almost 
bracketed second (the indices being 1:58, 1:52, and 155 
tespectively). 

When we come to Part II, mathematics gains pre-emin- 
ence for reliability in discriminating between Scholars and 
Commoners ; but again with history surprisingly close. 
And now classics drops—which we might expect from the 
fact that in some sections of Part IT in classics the kind of 
wotk differs so greatly from the largely linguistic work 
involved in Part I. The figures for Part II are: 


SUPERIORITY INDICES OF SCHOLARS xo CoMMONERS IN Parr П 
е 


1. Mathematics . Р n . 54 No. of Scholars . . 73 
2. History . x Я Р . 192 No. of Scholars .  '. 26 
[3. Modern Languages e. 0 r4] No. of Scholars . ^. 14 
4. Classics . Е Р К м 137 No. of Scholars . . 74 
5. Natural Sciences R 130 No. of Scholars . - 55 


1 It will be seen that the number of Scholars was very small in modern languages 
and history. 


The superiority index in Part II, however, cannot be 
taken as indicating the reliability of a scholarship award in 
subject x as tested by degree examination in subject x. For 
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sometimes a Scholar in x, after taking Part I in x, proceeds 
to take Part II in another subject. Some history Part II 
candidates, for example, may have been Scholars in classics, 
and some candidates for the natural science tripos Part II 
may have been Scholats in mathematics or even in classics. 
Scholars taking Part II in mathematics and classics, how- 
evet, were almost exclusively scholars in those subjects 
alone. 


THE PERFORMANCE OF STUDENTS WHO TAKE UP A NEW 
SUBJECT For Parr П or THE TRIPOS 


As evidence of the dependence of scholarship awards on 
abilities specific to the subject in which the scholarship was 
obtained, it is of interest to consider the performance of 
students who, after taking Part I in one subject, take Part 
II in a different subject. On this I was able to obtain 
figures thtough the infinite patience of my colleague, Mr. 
W. G. Emmett, who traced in the university calendars over 
a period of six years the careers of Scholars, Exhibitioners, 
and Commoners who took Part I in one subject and Part 
П in another. The comparative results are given in Table 
XXIV, p. 158. 

Of coutse, one would expect that students who change 
over to a new subject in Part П would not do as well as 
those who continue with the same Subject ; and this appears 
clearly for each group—Scholars, Exhibitioners, and 
Commonets. But we ate interested now in the question 
whether the superiority of Scholars (or Exhibitioners) to 
Commoners remains as great as when all continue in the 
same subject. The superiority index of Scholars to Com- 
moners taking the same subject in Part П as in Part Lis 1:58 ; 
the superiority index of Scholars to Commoners taking 
а different subject in Part П is only 1-25. The number of 
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TABLE XXIV 
COMPARATIVE PERFORMANCE IN Parr II Trrpos EXAMINATIONS 


(а) Of men who took Part II in the same subject as Part I. 
(2) Of men who took Part П in a different subject from that taken in Part І. 





Taking same subject in Part IT Taking different subject in 
E asin Part I | Part II from Part I 





Performance in Part II Performance in Part II 








Percentage 








Scholars Class I E Class I 27 
Class П 6 ` -| Class H : 54 
Class Ш E Class ПІ 19 

Exhibitioners. | Class I . Class I 17 
Class П . Class II 61 
Class III D Class JIT 22 

Commoners Class I . Class I 55 
Class II . Class II 49°5 





Class Ш . Class IIT 45 








Exhibitioners taking Part II in a different subject (36) is 
very small as a basis for generalisation, but their results ate 
similat to those of the Scholars : the superiority index for 
Exhibitioners to Commoners was 1:27, whete the same sub- 
jects ate taken in Part II, but somewhat less (1:17) when a 
new subject is taken. 

As we have seen earlitr (Table XIX, p. 147), when stu- 
dents having taken Part I in one subject take Part I in an- 
other subject (instead of taking а Part II examination), the 
decided superiority of the Scholars to Exhibitioners in the 
first Part I almost disappeats when all take Part I in another 
subject (superiority index, 1-07). Again, the numbers were 
very small for generalisation ; but all the above sets of 
figures agree in suggesting that when, either in Part I or 
Part П, a new subject is taken after the first Part I (which in the 
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gteat majority of cases is taken in the subject on which the 
scholarship was awarded) then the superiority of Scholar to 
Exbibitioner or Commoner greatly declines ; and so does the 
supetiority of Exhibitioners to Commoners. This is com- 
prehensible if the excellence of performance in the various 
subjects at this advanced university stage depends to a con- 
siderable extent on abilities or interests specific to the pat- 
ticular subject itself, rather than on general ability or 
abilities which can be equally well adapted to any new 
subject. 

It is, of course, conceivable that there is some peculiar 
selective influence at work which would lead the weaker 
Scholats to change their subjects more frequently than do 
the weaker Exhibitioners ot Commonets, and the weaker 
Exhibitioners to change more frequently than do the weaker 
Commoners. But there seems to be no reason for thinking 
this. And in the case of the students taking Part I of two 
different triposes, we have the records of the two triposes 
for exactly the same students, and the reduction of the 
eatlier superiority of Scholars to Exhibitioners when a new 
Part I is taken is more striking in this set of figures than it is 
when Part П is considered. 

All the evidence, then, points to the great importance 
of abilities or interests specific to each subject for success 
in any of these tripos subjects. 


CONCLUSION 





Chapter XI 


MMARY OF RESULIS AND SUGGESTIONS. 


In view of the extensive ground we have covered, and the 
numerous items of evidence on both major and minor prob- 
lems raised, it may be useful to attempt a general survey of 
out enquity. 

In the first place, I should like to emphasise again the 
limits of out main problem. We have been concerned with 
the reliability of examinations as tested by subsequent per- 
formance at school ot univetsity—this being estimated by 
more complex and thorough examinations taken at a more 
matute age, and also by estimates of heads of schools based 
largely no doubt upon academic performance. Such cti- 
teria ate themselves by no means the only criteria of value. 
lt would have been of great interest if we could have 
followed our students still farther, and seen how they fare 
in after-life—as Dr. Schuster did in his enquiry as to 
university students (see p. 35). Мот is business or pro- 
fessional success the final test of education. To get ош 
ideal criterion we should have to find all that was of value in 
character and ‘culture’ in later life—a task which no 
psychological statisticiah is likely to attempt. 

To deny, however, that success in subsequent and more 
substantial examinations ig one valuable test of the prog- 
nostic reliability of an earlier examination, is to assume an 
attitude of complete scepticism as to examinations which 
goes beyond that of critical caution which this book 
advocates. 

Thete is one further general observation that may be 


made. Some may say that almost entirely we have been 
160 
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using жуйел examinations as ctitetia of success ; and they 
may legitimately urge that the ability to discuss a topic ot 
atgue a problem by word of mouth is as important as to be 
able to write about it. 

If in reply to this others contend that specific disabilities 
or inhibitions may hinder a candidate from doing himself 
justice in such oral discussion, may not the same also be true 
for written examinations, though no doubt to a lesser 
extent? 

So far, howevet, as such ctiticism (of the confining of 
ouf ctiteria to written tests) is valid, so far is it a further 
criticism of the whole system of awarding scholarships and 
degrees entirely or almost entirely on written examinations. 
To discuss the possibilities of avoiding this limitation, and 
the possibility or desirability of bringing in even qualities 
of character as a basis for the award of scholarships at uni- 
versities (or schools), has not been within the range of 
our enquity. 

Not has it been out concern to estimate all possible ad- 
vantages of examinations. To do this would need a long 
discussion of their value as a stimulus to many pupils, as a 
useful guide to some teachers, as a necessary test of attain- 
ments, and so on. ‘The contention here has been merely 
that there are certain grave weaknesses in the attempt to 
make estimates by examination even of the probable future 
academic success of the candidates ; and this is a question of 
great practical importance, especially in view of the many 
scholarships awatded in schools and universities. It is also 
argued that a weakening of the faith in the infallibility of 
examinations should lessen the insistence by business or 
professional men on certain paper qualifications, and so 
reduce to some extent the excessive pressure of examina- 
tions upon studies at school. 

1i 
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We mayı now summarise the main findings and pto- 
posals of this enquity, giving some further consequential 
suggestions not mentioned in the previous chapters. 
Though I have of course only been able to study samples 
of examinations, yet out enquity as to entrance examina- 
tions to secondary schools dealt with ten examination 
centres, and included the results of over 1,600 pupils in 
nearly sixty school classes. Also, the enquity as to 
univetsity scholarships dealt with five provincial univet- 
sities, covering over 2,000 students, and with five large 
men's and two women's colleges in each of the ancient 
English universities, comprising over 4,000 students. 


GENERAL CONCLUSIONS APPLYING TO’ ALL ‘TYPES OF 
EXAMINATIONS 


(т) Examinations аге liable to énstability in the sense that 
a different examination paper of the same type, or the same 
answer paper matked by a different examiner, may give an 
appreciably different result. 

(2) Such istability is partly due to ‘luck,’ in the sense 
that a particular set of questions happens to suit a particular 
candidate well or badly. This element of weakness can 
be met, but only partially, by avoiding too natrow a range 
of questions, and by giving adequate choice. 

(3) Another cause of such instability is varying * form’ 
in the candidates. This can partly be met by offering the 
possibility of a further examination either for all or at 
least fot border-line cases. 

(4) Instability may also be due to vatying standards of 
different examiners. This can be reduced to some extent 
by a discussion among examiners of standards and of 
markings, and by the checking of standards by a chief 
examiner; such work of the chief examiner, however, is 
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no guarantee against momentary lapses of an individual 
sub-examiner in his standard, and a chief examiner’s own 
standard is not absolutely constant. 

(5) The instability due to the four above-mentioned 
factots may vaty greatly in examinations of different kinds. 
In the text ate given examples of different orders given by 
similar examinations taken at about the same time, showing 
very little resemblance (especially when answers of an 
essay type are involved). On the other hand, some papets 
of the type used for entrance examinations to secondary 
schools may give remarkably similar orders, at least when 
marked by the same individual. Further research into the 
stability of examinations at all stages is urgently needed. 

(6) So far as the instability of examination results 
depends on varying standards of marking, the introduc- 
tion of mental tests or questions of a mental-test type 
greatly reduces such instability ; but this is not possible 
in examinations of a higher type—e.g. for university 
degtees. | 

(7) The prognostic reliability of an examination is greatly 
reduced when candidates have been trained by teachers of 
very vatying capacity, ог when some have been specially 
coached. ‘The influence of such factors is lessened by the 
use of mental tests. 


THe ENTRANCE EXAMINATIONS TO SECONDARY SCHOOLS 


(8) In the entrance examination to secondary schools 
here considered, the pupils at the border-line were very 
close together in marks. This means that, even with 
relatively ‘ stable? examinations, a repetition of the tests 
might have put a very different group of pupils among 
the ‘ just passed.’ 

(9) In most of the centres studied there was practically 


164 SUMMARY AND SUGGESTIONS (Chap. xr. 


no telation between the order of merit in the entrance 
examination and the order of metit at the end of the 
secondaty-school career. This greatly emphasises the 
uncertainty as to whether the right pupils were selected, 
not only neat the bordet-line pass mark, but within a wide 
range of it. 

(10) The unteliability of the entrance order will also 
lead to an unfair apportioning of scholarships or free 
places. 

(11) Detailed reports from Heads of secondary schools 
confirm these suggestions of uncertainty, by showing that 
a large number of pupils (about one-fifth of those in the 
| Schools studied) proved unsuited for the secondary-school 
course. 

(12) On the other hand, some pupils who only just 
sctape through the entrance test prove highly successful ; 
in a few instances, indeed, the last or nearly last becomes 
eventually the first of the entrance group. 

(13) In view of the great unreliability of most of the 
entrance examinations studied, and of the uncertainty 
neat the border-line of all, it is desirable that а second 
chance should be provided; but the standard for this 
second test should be slightly higher. 

(14) As the order of merit after one year in the secondary 
school proves a much more reliable prophecy of satis- 
factory performance at the end of the school career, it is 
desirable that the awards of free places and scholarships 
should be seriously teconsidered at this stage ; that those 
who have failed to justify their awards should be deprived 
of these, and that pupils who failed to get them at the 
entrance examination, but have done very well within the 
school, should now receive appropriate awatds. 

(15) The difficulty of ensuring a right selection for secon- 
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daty schools at 11+ also suggests that a way of entrance to 
secondaty schools should be open at a later stage; and 
also that transference from the secondary school to 
another type of school should be facilitated, or that a 
wider range of courses appropriate for different types of 
pupils should be available within the secondary school. 

(16) It should be emphasised that the failure of many 
pupils to succeed in the secondary-school course does 
not imply that they are necessarily unworthy of some type 
of further education after fourteen years. The right 
adjustment, it is suggested, for such is to transfet them to 
central or senior or technical schools, or to modify the 
curriculum for them in the secondary school. 

(17) In the entrance examination in one centre an appre- 
ciable improvement was found to follow the introduction 
of mental tests and a systematic scheme for the marking of 
essays and other answers. Results, however, gathered in 
this enquiry as to the use of mental tests confirm those 
previously published in suggesting that the tests at present 
in use do not alone give much mote reliable estimates of 
future school successes than do the most carefully planned 
entrance examinations, though they are valuable as a 
supplement. 

(18) It is suggested that one of the main reasons why the 
entrance examination and the present mental tests fail is 
that they do not detect the presence or absence of specific 
abilities needed for the wider curriculum of the secondary 
school. ‘This is confirmed by the fact that once the 
pupil has been tested in all the branches of study in the 
secondary, school—by the end of the first year, and especi- 
ally by the end of his second—the order of merit is very 
steady from year to year. The further development of 
tests of specific abilities, e.g. for foreign languages, or 
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mathematics, may in the future be of considerable service 
in selecting the pupils more reliably for secondary-school 
work. 

(19) That the changes from entry order to order at 
School Certificate stage are not due to mere passing of time 
and different rates of development of different pupils is 
shown by the fact that there is a change between entty 
otder and order at the end of the first year greater than 
the change between the order at the end of the first year 
and order at the end of the fourth year. 

(20) Definite evidence has been given showing that 
the great resemblance between orders of metit from yeat 
to yeat within the secondary schools studied is not due 
appreciably to the teachers! knowledge of the pupils. 

(21) It is suggested that mote exact observations should 
be made as to the prognostic value of the tepotts of the 
Heads of elementary schools on their own pupils, and that 
special weight should be given to the reports of those Heads 
who are found to be specially reliable. 


SCHOOL CERTIFICATE EXAMINATIONS 


(22) As to the School Certificate, the dependence of the 
gaining or loss of the Certificate on a few marks more ot 
less in one subject introduces grave anomalies. A more 
elastic (but steeply gtaded) scale of compensation is 
suggested. At present a boy A with 50 per cent. more 
marks than B (on the total) may fail to get a Certificate 
while B gains it. 

(23) Though the mere gaining or loss of a School 
Certificate may give a most unreliable estimate of the 
pupil, the relation between the order of merit in the total 
matks in the School Certificate examination and in the 
schools’ own examinations at the same time is a very close 
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one. This also suggests that relatively more weight should 
be given, in some School Certificate examinations at least, 
to the total marks in determining the award of the Certi- 
ficate. 

(24) As is already recognised, special uncertainty 
attaches to the marking of answers of an essay type. But 
it is not usually known how possible it is for a candidate to 
write correct English for the moment in an essay test, 
while thinking chiefly of the importance of correct writing, 
and yet to lapse into his natural bad English on other 
occasions. From this point of view, marks for English 
in all papers in a School Certificate examination would 
seem to be a sounder test than a separate essay paper; if it 
be argued that subject examiners will not pay adequate 
attention to English, or that their estimates cannot be 
equated, it may be considered whether a special examiner 
in English composition may not see the papers in all or 
most subjects, to assess the candidates’ capacity for writing 
English. 


ScHOLARSHIP AWARDS IN PROVINCIAL UNIVERSITIES 


(25) As to scholarships in the provincial universities, 
neatly two-fifths of the awards examined were given to 
students who proved unworthy of them, if we take as our 
criterion the attainment of at least second-class honours ; 
and one-third of the Non-scholars beat two-fifths of the 
Scholars. | 

(26) On further analysis, it proved that the weakness 
was chiefly due to scholarships not awarded by the univer- 
sities themselves. Of the holders of these, nearly a half 
prove unworthy of their awards. Of Scholars selected by 
the universities themselves the proportion is one-fifth. 

(27) Men university Scholars gain first-class honours 
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twice as often relatively as do women Scholars. But the 
proportion of women who fail to reach the standard of 
second-class honours is practically the same as that of the 
men. 

(28) The superiority of the men to the women Scholars in the 
gaining of first-class honours is not, as sometimes thought, 
due to the special weakness of women in science. The 
superiority shows itself even more in the Arts subjects. 

(29) In comparing the performance of Scholars in Arts 
with Scholars in Science, it is important to bear in mind that, 
while firsts are more frequently awarded in Science, there 
are also proportionately more failures and pass degrees in 
Science than in Arts. 

(30) The superiority of Scholars to Non-scholats is 
slightly greater in Science than in Arts, but not sufficiently 
to warrant our saying that scholarship awards in Science 
are mote reliable than scholarship awards in Arts. 

(31) One aspect of the reliability of examination 
awards is the comparative ease of gaining a first in different 
subjects. The difficulty is pointed out of defining exactly 
what is meant by saying that the standards in two different 
subjects are equal; and it is argued that any comparison 
requires a reference to the degree of general ability involved 
in both subjects, and to the relative frequency in the 
whole population of the specific abilities involved in the 
two subjects. 

(32) In view of the great unreliability of the scholarship 
awards in these provincial universities, it is suggested : 

(а) That fewer entrance scholarships should be given, 
and that more internal scholarships should be 
awarded after one or mote years in the university. 

. (b) That the bodies (other than the universities) who 
awatd scholarships should model their procedure 
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(in selecting) more on that of the univetsities, or 
should hand over the selection to the universities 
themselves. 
In any case, further research into the reliability of 
individual school or county or borough scholarship 
awards seems highly desirable. 


Tue AWARDS OF STATE SCHOLARSHIPS 


(33) If we use the same criterion as before, about 
one-tenth of the State Scholars fail to justify their awards 
—and are beaten, at the provincial universities, by 
one-third of those who have no scholarships at all; 
while half the State Scholars in modern universities ate 
beaten by about one-tenth of the Non-scholats. This too 
was before the number of State scholarships was increased. 

(34) Among State Scholars at universities the men do 
decidedly better than the women. Again, this is not due 
to supetiority of the men in certain subjects. The men 
gain a higher percentage of ‘firsts? in every subject 
(including modern languages) except English, in which 
the percentages are the same. This result would seem to 
justify the recent change made by the Board of Education, 
by which the women are not necessarily given as many 
State scholarships as the men. 


ScHOLARSHIP AWARDS AT OXFORD AND CAMBRIDGE 


(35) The awards of scholarships at Oxford and Cam- 
bridge seem definitely more reliable than the awards (taken 
en bloc) at the provincial universities ; in that the superiority 
of Scholars to Non-scholats (even to the picked ones who 
take honours degtees) is greater. This superior reliability, 
however, is reduced if we consider separately, as we should, 
the Scholars elected by the provincial universities them- 
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selves. Even thus, however, a smaller proportion of 
Scholars and Exhibitioners at Oxford and Cambridge fail 
to justify their awards (by getting at least a second class)— 
about one-tenth, against one-fifth of the men Scholars in 
provincial universities. (The Oxford figutes are based on 
the assumption that one-third of the third-class honours 
awards ate counted as equivalent to ‘seconds’ at Cam- 
bridge and elsewhere.) 

(36) At Oxford, however, the distinction between 
Scholars and Exhibitionets seems insecure. At Cambridge 
the discrimination between Scholars and Exhibitioners in 
the entrance scholarship examinations seems much more 
teliable. These statements remain true, even if one 
eliminates one Oxford and one Cambridge college at which 
the Exhibitioners do extremely well—nearly as well as 
Scholars of the other colleges here studied. 

(37) At Oxford, one-fifth of the scholarships and exhi- 
bitions were ‘ misplaced,’ in the sense that they might 
better have gone to Commoners who got ‘firsts ’ instead 
of to Scholats and Exhibitioners who got ‘seconds’ or 
something lower. The proportion is almost exactly the 
same at Cambridge if we consider the Tripos examination 
Part П as the best test. 

(38) The poor performance of some Scholars in the 
final honours examinatiens is due in some cases to break- 
down in health, or to weakness in some qualities of 
character. It seems doubtful, however, whether such 
causes can account for so large a section doing badly ; and 
again it looks as though errors of selection are made at the 
scholarship examinations, perhaps inevitably. Unequal 
coaching is a disturbing factor here also; and some 
students excellent at school work prove inferior in the 
different and freer studies of the university. 
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(39) In view of the excellent later petformance in the 
Cambridge Tripos Part II of students who gain a first in 
Part I, it is suggested that it would be safer to award more 
internal scholarships on Part I results instead of exhibi- 
tions given on entrance scholarship examinations. Pos- 
sibly a similar principle could be adopted at Oxford. 

(40) It is notable that where Scholars and Exhibitioners 
change their subject, after taking a Part I tripos examination 
at Cambridge, and take Part I of a different tripos, the 
Exhibitionets do practically as well as the Scholars in the 
new ttipos ; and when Part II is taken in a new subject the 
superiority of Scholars to Exhibitioners is also greatly 
reduced. ‘These facts point to the importance of specific 
abilities for a high performance in entrance scholarship 
examinations and in university examinations in the same 
subjects. | 

(41) As tegatds women students at Oxford and Cambridge, 
here the lesser supetiotity of Scholars to Commoners 
is possibly partly due to a keener competition among 
women even for entrance at Oxford and Cambridge. At 
Oxford, in the two colleges studied, the discrimination 
between Scholars and Exhibitioners is quite insecure, the 
Exhibitioners (for the six years taken) actually doing slightly 
better than the Scholars in the final honours examinations. 
At Cambridge this discriminatioh is sounder. Yet even 
there, the superiority of Scholars to Exhibitioners is only 
slight. 

(42) The relative reliability of scholarship awards in different 
subjects at Cambridge was tested by estimating the superi- 
otity of the performance of Scholars to that of Exhibi- 
tioners and Commoners in the several subjects. In some 
subjects, notably history, the unreliability is not as great 
as has been thought, relatively to others. On the whole, 
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the awards in mathematics ate the most reliable, with 
histoty and classics bracketed a very close second. 

(43) The relative frequency of the awards of * firsts” at 
Cambridge is much greater in some ітіроѕеѕ than in others. 
Firsts ate mote ftequently given in classics and mathematics 
than in any other ttipos. This is true both for Part I and 
for Part II examinations. The percentage of Firsts is 
extremely low in economics, English, law, and histoty. 


In conclusion, I should like to protest strongly against 
any general inference'from this enquiry that a reduction in 
expenditure is justified, either on secondary education (or 
at least on education after 14 years) or on scholarships to 
modern or ancient universities. 

As indicated in the text, vatious other alternatives are 
possible. Indeed, the enquiry shows that there are many 
pupils not obtaining entrance or scholarships to secondary 
schools, or scholarships at the universities, who would 
be likely to do as well as many who at present gain these 
awards and thoroughly justify them. 

Furthermore, we may expect further research to improve 
out methods of selection at all stages. Indeed, much 
might immediately be done if some practical way could be 
found of readjusting erroneous selections shortly after 
entry to the secondary schools—and after one year in the 
university, ot by changing some entrance scholarships 
into internal or post-graduate scholarships, especially at 
the provincial universities. 

If further enquiry shows that some university scholar- 
ships tied to particular schools or awarded by local educa- 
tion authorities are continually being awarded to Scholars 
who prove unsatisfactory, the question may well be con- 
sideted whether such awards should not be made by 
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methods in which a wider basis of comparison can be 
obtained (as with provincial university scholarships proper, 
or college scholarships at Oxford and Cambridge), and so a 
sound minimum standard of attainment required; and 
thus, if a candidate of sufficient merit was not forthcoming 
in any year, the scholarship might be withheld and reserved 
to give a much-needed post-graduate scholarship to the 
next Scholar from that school or town who should dis- 
tinguish himself in the final degree examinations. 


APPENDIX I—TO PART П 


ENTRANCE EXAMINATIONS TO SECONDARY SCHOOLS 


Att correlations here and elsewhere were worked by Spearman’s 
mote accurate ‘ rank ’ formula (о). 


I (а) CORRELATION OF ENTRANCE EXAMINATION WITH SCHOOL 
CERTIFICATE EXAMINATION, Centres А, В, C, D, AND Е 


For each centre the correlations of the various years given 
were first averaged. These average correlations for individual 
centres wete as below; these averages were then themselves 
averaged, giving 0-03. This is only a rough average, for greater 
weight should really be given to those correlations based on the 
greater numbers of pupils. As will be seen below, a true aver- 
age would give greater weight to the figure for Centres B and C, 
and less to D, slightly raising the average. 


For Centre A. . $ . .—o'15 (The average for two examinations 
in successive years (1924 and 1925), 50 
pupils in all.) 

For Centre B . . : . O15 (The average for four examinations 


in four successive years (1921-1924), 
81 pupils in all.) 


For Centre C : ‘ . . o05 (The average for three examinations 
| 1924-1926, 66 pupils in all.) 

For Centre D 3 ` & . ооо (The average of two classes in the 

examination of 1925, 22 pupils in all.) 

For Centre E . . . . ot (The average of three examinations, 


- 1925-1926, 36 pupils in all.) 


The numbers of pupils in the different years for any given 
school were almost the same, except in C, where the numbers and 
correlations for the three years were : (1) 19 pupils, p = o'19; 
(2) 28 pupils, р =0-00; (3) 19 pupils, р = —o-o5. It will be seen 
that giving the greater weight due to (2) would lower the average 
correlation for Centre C. 

| If it be objected that the correlations given above refer to. 
highly selected groups, it may be pointed out that the high correla- 
174 
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tions of School Certificate examination with school order after 
four years (about o-8), and of school order after one year with 
School Certificate after four years (viz. about o'7), also refer to 
similar selected groups. (See note on Selection, Appendix I (f), 


p. 184.) 


I (0) CORRELATION OF ENTRANCE EXAMINATION WITH SCHOOLS’ 
OWN EXAMINATIONS, CENTRES А, B, C, AND E 
Correlation of entrance examination with secondary schools' 
own examinations four or five years later. Averages for same 
years and classes given in Appendix I (a). 


Centre A. . Р . . : : Е .—0'05 
Centre В . . : s é 5 . . O09 
Centre С . " . 2 Е А 2 . оол 
Centre В. А s И б . . . O24 


All these school orders were based on the schools’ own examina- 

tions held just about the same time as the School Certificate 

examination, except that in C the marks fot work throughout the 

year counted equally with the schools' own examinations at 
' Christmas and later. 


I (г) Nore ow Рковавін Errors ron Cenrres А, В, C, 
D, Е, H, AND I 

In reference to the correlations for Centres A, B, C, D, and E, 
the coefficients were negative, or so extremely small that it was 
not worth while giving probable errors. 

In Centre H the P. E. of the correlation of entrance order with 
School Certificate order (0-39) for the 24 boys would be some- 
thing over 0-12. Following the tequitement that to indicate a 
genuine relationship a correlation coefficient for 50 cases should 
be more than three times the P.E., and that when the number is 

‘less than 30 it should be still greater in proportion, we see that 
this coefficient is very doubtful. The correlation of об for the 
girls—15 in number—is still less certain. The usual formula 
would give a P.E. of about o-12, but as the number of pupils was 
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only 15, the P.E. is much higher. (See W. Brown and G. H. 
Thomson, Essentials of Mental Measurement (1921), p. 103.) 
Thus it is not certain that even the correlation of о.б in this 
case indicates any essential relation between success in School 
Certificate examination and success in the entrance examination ; 
but the average of the two positive correlations (boys and girls) 
justifies the concession made in the text that this Centre H may 
claim some degree of resemblance between entrance order and 
otder at School Certificate. 

In Centre I, similarly each of the correlations is in itself very 
unreliable. But the average of the six correlations, each group 
of three based on 40 pupils, suggests a slight positive correlation 
for Centre I. 


I (d) MATRICULATION As A TEST or SUCCESS 

‚ А comparison is given below of the performance in Centre J 
of top, middle and bottom groups at entrance as regards the 
gaining of School Certificate and Matriculation. School Certi- 
ficates gained in four yeats were compared with matriculation 
certificates gained in four or five years because only thus could 
similar numbers and percentages be obtained. 








CENTRE J 
School Certificate gained Matriculation gained in Four 
in Four Years or Five Years 
Top groups i Я Р 142/284 = 50:09 121/284 = 42-69%, 
Middle groups . š . 102/286 = 35:6% 81/286 — 28:394 
Bottom groups . . «| 0 63/284 = 22:29 42/284 = 148% 








In this table the relative superiority of top group to bottom 
group at first sight appears greater in the matriculation figures 


than in the School Certificate, the data being ЖЕ Hs , Ot 2:88, for 


matriculation, as compared with 5 555, ог 2:25, for School 
Certificate. 
On the other hand, the failure to gain the awards may with 
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equal validity be taken as the basis of comparison in the two cases, 
for those who fail are all classed in one rank just as are those who 
pass, and a measure of relative superiority can thus be obtained 
from the inverse ratio of failures in the two groups considered. 
From this standpoint we find that 57-4 per cent. of the top group 
and 85:2 per cent. of the bottom group fail to get matriculation, 
85:2 2 
ae 
while 50-0 per cent. and 77:8 per cent. respectively fail to get 
School Certificate, a relative superiority of 1:56; so that regarded 
in this way Matriculation turns out to be a Æss reliable criterion 
of success than School Certificate. This indicates that the 
attempt to compare the value, as criteria, of Matriculation with 
that of School Certificate awards by either method indicated 
above is not satisfactory. 

Apart from these considerations, however, there are a number 
of inherent factors which might conceivably bring about a closer 
connection between performance in the entry examination and 
the gaining of matriculaticn than between performance at entry 
and School Certificate awards. 

In the first place, it has been suggested that such closer connec- 
tion is to be expected on general principles of correlation ; if of 
the successful candidates for entrance examination the School 
Certificate winners are more nearly jo per cent. of the whole 
than are the winners of matriculation, then the matriculation 
results should correlate more closely with the entrance examina- 
tion than do the School Certificate resylts.? 

Again, pupils low down at entry would tend to have no 
maintenance grant or even free place, and so would be less 
likely to stay on for a fifth year ‘to gain matriculation if the 
School Certificate examination is passed in four years, though 
they may be really abler than others above them in the entrance 
order to whom grants were awarded. 

Further, there is a danger that a greater scatter of marks in an 
“1 Mr. Frank Sandon makes this suggestion, and expresses his opinion that this is 
the explanation of the apparent difference. 

12 





a relative superiority of top to bottom group of 
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atithmetic paper as compared with English will give undue weight 
to arithmetic if its marks are added without modification to those 
for English in order to produce the order of merit in the entrance 
examination. Now, for all those who do not take Latin, mathe- 
matics is compulsory in most matriculation examinations : and a 
good many pupils who do take Latin.rely on mathematics to 
provide the essential credit in the Science group of subjects. 
Hence the special importance of mathematics for matriculation 
as compared with School Certificate may provide a closer link 
with the entrance examination where arithmetic is emphasised. 
For although the abilities for arithmetic, algebra, and geometry 
are by no means identical, they are more closely allied than say 
arithmetic and languages, and, still more important, the mathe- 
matics in the matriculation examination includes arithmetic. 

Then again, as suggested in the text (Chapter VI), the entrance 
examination in English, arithmetic, and intelligence tests gives 
great scope for the display of general intelligence, but little for 
specific abilities required for some secondary-school subjects. 
Now in matriculation a credit standard is required in five sub- 
jects, representing at least three different groups. It is possible 
that general ability becomes of greater importance relative to 
specific abilities, in gaining the higher positions in various 
subjects: that is, that to score between 50 per cent. and бо per 
cent. in a language or a Science depends relatively more upon a 
general function or functions than does the attainment of a mere 
pass at 40 per cent., for which some specific abilities may com- 
pensate more for weakness in a general function or functions. 
Ido not say there is any evidence for this; I merely suggest it as 
a possibility. Obviously much will depend upon the type of 
questions set—the presence or absence of a few questions 
requiring a high degree of a general function (whether one * g 
or a wide group factor). 

Again, it is also possible that certain compulsory subjects. 
for matriculation (Latin or mathematics) may themselves require 
for success relatively greater general capacity than do the other 
subjects. But the evidence on this point is so far conflicting. 
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Finally, it must be borne in mind that even if the entrance 
examinations had been very reliable in predicting success in 
matriculation of the very pick of the entrants, that would not 
meet our main criticism that the entrance examinations are weak 
in discriminating low down in the pass list and especially near the 
border-line of failure. 


І (e) CORRELATIONS OF ORDERS IN ENTRANCE EXAMINATION 
WITH SCHOOL CERTIFICATE EXAMINATION IN CENTRE J 








Year of Number of! Correlation 
Length of Course | Pupils | Coefficient 

































































Entry 
1923 4 years 18 0:58 
School x (boys) | 1924 4 years 23 0:36 
1925 4 years 34 0°39 
1924 4 years 18 0°39 
1924 5 years 27 0:51 
School 2 (boys). $ 1926 4 years 30 0'46 
1927 4 years 21 046 
1925 4 years 19 0°33 
School 3 (boys). 1924 4 years 24 0:27 
1924 5 years 19 0-18 
1923 4 years 20 032 
School 4 (girls) | 1924 4 yeats 24 Ol 
1927 4 years _ 23 044 
School 5 (boys). a : 1925 4years — 19 ‚ 0128 
School 6 (girls) А r 1924 | Syears | 19 | O44 — 
1923 5 years R 21 019 
1925 4 yeats 33 0:48 
School 7 (boys). | 1926 4 years 18 o'65 
1927 4 years 20 0:12 
School 8 (boys). ^ . : 1924 5 years 23 0°43 
А 1926 4 yeats 18 O'51 
School 9 (girls) { 1926 . $ years ò 039 
1926 5 yeats 1 обо 
School то (boys) . 1927 4 years 21 0-68 
School 11 (girls) 1926 4 years 46 0°35 
School 12 (boys) . . 1926 4 years 45 045 
Boys Я . . I9 classes — 455 pupils. Rough average correlation, 0:41. 
Girls E . . 7 classes = 170 pupils. Rough average correlation, 0°36. 
Tota  . A . 26 classes =625 pupils. Rough average correlation, 0-40. 


i2* 
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I (f) CORRELATION OF ‘ ENTRANCE ORDERS’ WITH ' BLENDED 
ORDERS? or SCHOOL CERTIFICATE AND SCHOOLS’ Own 
EXAMINATIONS IN CENTRE J 





Corns. of | Corrlns. of | Comins. of 









































h entrance 
ў queue | uem | onderwith 
Course Sch. Cert. | school order} сс and 
5 . order. at S.C. stage School order 
School 5 4 yeats 0°33 058 · 0°52 
4 yeats 0'27 orks 0°23 
5 yeats o'I 0'29 0°23 
School 4 . 4 years 0°53 050 
4 years 0°07 0:06 
4 years 0:54 0:46 
Schools . i 4 years 0'4X 040 
Schoodló , 5 years 0:16 031 
School; . 5 years 0°09 o'15 
Schoodl8 . 5 years | 23 0:32 0'39 
Schoolg . ‹ $ yeats | 20 0°43 0:43 
School зо . 5 years | 18 0:60 0°59 
4 years | 22 0*79 0775 
School 12 . Е 4years | 45 0°45 047 
Average for 316 pupils 0:39 0°39 








Note.—These classes were not, of course, specially selected. They were the 
only ones for which I obtained figures for both School Certificate and the schools’ 
own orders, each based on 18 or more pupils. 


I (g) CORRELATIONS BETWEEN ORDERS, IN A Нісн SCHOOL, OF 
Boys oF 12 To 13 YEARS AND THEIR ORDERS AT 17 TO 18 YEARS 


Order at age 12-15 yearl, compared with order at 17-18 years : 
1924 to 1929, r = 0-77 (14 boys). 

Order at age 13-14 years, compared with order at 17-18 years : 
1924 to 1928, r = 0:79 (19 boys). 
1925 tO 1929, r = 0°80 (22 boys). 

Order at age 14-15, years, compared with order at 18-19 years : 
1926 to 1930, r = 0-78 (19 boys). 

The method of procedure was as follows : the boys of 12 years 

were found distributed in four forms—e.g. 1st, 2nd, and 3rd 
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(lower and upper), the order in each form being based on 
school examinations and the work of the year, all subjects being 
included, except mathematics. Each form order was now 
divided into three. This gave twelve different groups or posi- 
tions, and each boy’s group was noted. A similar procedure was 
followed for the same boys when they were 17-18 years old, 
these classes being Upper VI (which counted as one group only 
and was not divided), Lower VI, Upper V, and Lower V. "This 
gave ten positions or groups. The number of boys in each of 
these forms was small, only one group containing over 20 boys. 
But the steadiness of the correlation about 0-79 is noticeable, and 
the total number of boys on which the average was based was 
74. (For this correlation the Pearson Product-Moment formula 
was used.) 


1(4) CORRELATIONS OF (4) ORDER AT END OF ONE YEAR IN 
SECONDARY SCHOOL WITH ORDER IN SCHOOL CERTIFICATE 
(CENTRE J), COMPARED WITH CORRELATIONS OF (b) En- 
TRANCE ORDER WITH SCHOOL CERTIFICATE 
(THE NUMBER OF PUPILS IN EACH CLASS IS GIVEN IN PARENTHESES) 





Year Length Corrln. of sch. order Corrin. of order at 
of 



































of at end of one yr. with | entrance with order 

entry course order in Sch, Cert. in Sch. Cert. 
School 2 1924 | 4 years 0°46 (18) 0°39 (18) 
1926 | 4 years 0°68 (30) 0°46 (30) 
1927 | 4 years o'69 (21) o46 (21) 
School 3 1924 | 4 years o71 (24) o27 (24) 
1924 | 5 years o'44 (19) o18 (19) 
School 4 1923 4 yeats 73 (20) 0°32 (20) 
1924 | 4 years 76 (24) o11 (24) 

1927 | 4 years o47 (22) o44 (23)! 
School 1925 | 4years (В) |, 087 (23) o24 (23) 
E d 196 4 yeats o82 (18) o65 (18) 
1927 | 4 years 0°68 (20) o12 (20) 

l-— " 
School 9 1926 | 4 years 0:68 (18) osi (18) 

1926 | 5 years o76 (19) 039 (20)? 

School то 1927 4 yeats o84 (22) 0:68 (24)? 
Average 0:69 (298) 0°37 (302) 





1 Certain pupils in these School Certificate classes were absent from the school 
examinations at the end of the first year. 


182 APPENDIX I (7) 


I (7) AGE ALLOWANCES IN THE ENTRANCE EXAMINATIONS 


In an examination with an age limit of say 11; 0 to 1250, it is 
obviously unfair to compare the examination marks of a child 
of тт years т month with those of a child of тт years тї months, 
without giving the younger some age allowance. As an 
example of the effect of giving no age allowance, we may quote 

‘the results of an examination in a large city. In the 1923 entry 
for the preliminary examination in Manchester, the following 
numbers of candidates of different ages reached a 50 per cent. 
mark : ` 


Age 11:11 А ; а А , . 451 
Age 11:9 ` ; Я . è . 331 
Age 11:6 2 2 > А . 234 
Age 11:3 . Я . К : . 162 
Age 11:0 108 


(The numbers for intervening age months were similarly graded, 
with one slight exception.) 1 

In order to make up to the younger child for his shorter time 
at work which prepares him for the examination, increasing 
numbers of centres give age allowances at their entrance examina- 
tions—only two of the centres we studied did not. 

The intention is presumably so to adjust the age allowance as 
to give the 11-year-old an equal chance of competing with a 12- 
year-old of the same ability. If the age allowance compensates 
the 11-yeat-old adequately for his one year less schooling, then if 
he is an abler boy than the {2-year-old he will beat the 12-year-old 
and take his place in a vacancy in the secondary school. 

In our enquiry the question at once arose : if there has been 
an age allowance for the order at entry, should we compare such 
an order with an order at School Certificate where there is no 
age allowance? І discussed this question with a group of four 
Headmasters, and we were finally unanimous in thinking that it 


1 Reported on p. 200 of Examinations in Public Elementary Schools (Report of the 


Joint Committee of Associations of Education Committees and the National Union 
of Teachers, London, 1930). 
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was proper and fair to keep the age allowance at entry in spite 
of there being none for the order four or five years later. The 
reasons which influenced us were as follows : 

(1) A difference of twelve or even six months at about 11 
yeats means a big difference in the length of tuition in the sub- 
jects of the examination in arithmetic and English ; whereas all 
statt level within the secondary school as regards the new 
subjects ; and apart from that, six months more schooling is 
telatively less when two boys of 16 are compared (after say ten 
years at school) than it is when the boys are only 11—after five 
years at school. 

(2) Apart from length of school life, six months’ greater 
maturity at 11 is of relatively greater significance than it is at 16. 

(3) In addition, the younger set of boys would often be grouped 
together for a five-year course while the olderones take afour-yeat 
course. This was the case in a considerable proportion of the 
classes which came under our view. 

As a check on out decision, however, I obtained the entrance 
order without age allowance in nineteen classes (representing four 
of the centres), and then found the correlation of these orders 
with the orders in School Certificate. It averaged 0-32. For the 
same classes the correlation of School Certificate and entrance 
order with age allowance was o-31. In two centres the correlation 
was slightly lower when age allowance at entry was ignored ; in 
one (the largest) the correlations were about the same ; in one 
centre the correlation was slightly higher without age allowance 
—but this was based on only one class. 

Also in two other of our eight cehtres no age allowance was 
given; so we may in any case safely regard our results as un- 
affected by the inclusion of age allowance even if one does not 
agree with the view expressed above—namely, that it is fairer to 
include the age allowance at entry. 

Presumably the Heads of elementary schools would only 
recommend for admission to the entrance examination younger 
pupils who were thought at least as good for their age as the 
older ones. ‘The tendency to present only pupils in given 
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classes would send forward young pupils who were really better 
(for their age) than the older. Тһе correlation figure, however, 
suggests that such selection and pressing forward of young 
pupils of higher I.Qs. has not gone too far in the centres 
studied. For if the youngest at entrance were naturally much 
© cleveret? they would gradually move up in the order of metit 
— passing some of the older who beat them in the entrance 
order when age allowance was ignored. This would tend to 
bring the order of merit at four years nearer to the order of metit 
at entrance with age allowance. 

In comparing the order at entrance with order after one year 
only, the question of including age allowance is somewhat 
different. ‘The extra six ot even twelve months of maturity is 
relatively much greater at 11 years: the extra schooling before 
entry to the secondary school has also greater weight, though 
some important subjects at the secondary school are entirely or 
largely new to the pupils. Hence, in comparing entrance order 
with order after one year we have given (p. 87) correlations both 
with age allowance and without age allowance. 


I (/) NOTE on THE EFFECT OF SELECTION ON CORRELATIONS 


If 1,000 children ate well tested for intelligence, and 30 
selected who are all of exactly the same I.Q., then in English and 
arithmetic tests these 30 would give no positive correlation, if 
the fitst selection by intelligence secutes that a general (or group) 
factor is found to an equa] degree in each of the 30; and if 
special ability in arithmetic tends to be given by a kind provi- 
dence to those weak in English (and vice versa), there would be 
a negative correlation between the arithmetic and English, even 
though both may involve a general factor (or group factot) to 
a high degree. 
| But the kind of selection that occurs for entrance examinations 
is very different. At mostall we do is to pick out the best, say 
100 out of 500 candidates, or perhaps 2,000 out of 10,000 possible 
candidates, based on simple examinations, or examinations plus 
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mentaltests. The fact that these are by no means even apptoxim- 
ately equal in general intelligence is shown (a) by great difference 
in performance in mental tests of those just getting into any given 
secondary school (e.g. the range of scores for the intelligence tests 
for Centre J for 113 pupils recommended for admission into a 
typical school was from ror to 45 in a typical year, the maximum 
being 120); (b) by the fact that high correlations are given by 
these pupils between two examinations in the second and third 
years in the secondary schools, and even between School Certifi- 
cate order and schools’ own examinations in the fourth and fifth 
yeats, when the process of selection has been carried farther by 
the lagging behind of those not able to reach the School Certifi- 
cate class; and (г) by the high correlations between performance 
in school after one yeat with performance after four ot five years. 

Again, we found that even in the centres where the correla- 
tion between entrance examination and School Certificate order 
was practically nil, the correlation between school order in the 
fourth year and School Certificate was very high—about o°8. 
Yet this was after further selection in some cases by division into 
four-year School Certificate course pupils and five-year course 
pupils, and after previous selection had taken place by the 
dropping out of some.weaker pupils. 

Finally, even if in any entrance examination it proved that the 
pupils selected for the secondary school were all of about the same 
capacity as judged by the entrance tests, we could have no 
confidence that those below this mark would be appreciably 
weaker, ot that the examination was really serving its purpose." 
And not only may some below the ldwest level for admission be 
as good as those just above, but, pupils approximately equal in 
capacity would be receiving, some free places and money awards, 
some free places only,,others admission on payment of fees, and 
others admission subject to a further oral examination. 


1 The influence of selection is fully dealt with in The Essentials of Mental Measure- 
ment, by W. Brown and Godfrey Thomson, chapter vii. 
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1 (А) Tue RELIABILITY OF THE ENTRANCE EXAMINATION TO A 
Junior TECHNICAL SCHOOL 


As this book goes to. press results are received from a junior 
technical school in another centre, which so far as they go con- 
fitm those given for Centres A to J. 

The entrance examination to this technical school included 
arithmetic, English, a general knowledge paper, and in addition 
geometrical drawing. 'The boys wete aged 13 yeats, and no 
age allowance was given. The papers wete set and marked by 
the staff of the school. 

. А comparison has been made between the order of merit in 
the entrance examinations (for 1930 and 1931) to this technical 
school and the school's own orders of metit at the end of the first, 
second, and third terms. 

The orders at the end of each term were based on the class 
work and the terminal examinations, in which the subjects were 
mathematics, Englísh, technical drawing, history, geography, 
woodwork, metalwork, and German. 

The correlations obtained were as follows (the numbers of 
pupils are shown in parentheses) : 





Correlation of Order at Entrance with Order at end of: 














First Term Second Term Third Term 
Group A ; 5 0:76 (22) о‹67 (22) o64 (22) 
Group B › i 0°54 (24) 0°43 (24) 0°49 (24) 
Group C > : о'42 (22) 0:53 (22) o'44 (22) 
Group D А " oo6 (22) 0°36 (23) 0°39 (23) 
Group E s A oœ43 (23) o20 (23) o9 (23) 
Average " : 0°40 (1 13) 044 (114) 0'45 (114) 





The correlations werte calculated by Spearman’s footrule method (R), and 
converted to the product-moment coefficients (r) by means of a table. da 

It will be seen that the average correlation of the order at 
entrance with that at the end of the first year (third term) is 
only 0-45 for 114 pupils, a figute which is practically the same 
as that obtained for Centre J, viz. o-46. For the four groups 
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A, B, C, and D, the orders for the fourth term were also obtained, 
and the average correlations of these with entrance order was 
practically identical with the average correlations for the second 
and third terms. 

This entrance examination then proves no mote reliable than 
some of the entrance examinations to secondaty schools, although 
the candidates are somewhat more mature. | 


APPENDIX II—TO PART III 


PERFORMANCES OF SCHOLARS AND NON-SCHOLARS IN THE 


П (a) PROVINCIAL UNIVERSITIES 


UNIVERSITIES DEGREE EXAMINATIONS 


Perfotmance of Scholars and Non-scholars (Percentages only) in Individual 
Universities (Arts and Science Combined) 























































































































Universities 
All Uni- 
Class A B c LE norm нае 
ошу) 
Scholars H 252 | зот | 26:1 27:4 34 27:4 
П | зто | 313 | 348 | 321 | 4 346 
IH 8-4 3:I T4 10°1 2 ТІ 
Pass 19:5 | 28:8 | 297 | 244 16 25:6 
Fail g'o 61 2:0 Go 7 5°3 
тоо'о | IOO'O | тоо'о | 1:000 | тоо | x000 
No. of students 
represented 190 163 299 168 44 864 
Non-scholars'’ I 8:4 8-0 8-8 6-6 9 8:5 
I 276 | 31:8 18:6 | 268 22 249 
ш T5 g'o 63 | 1r6 5 T9 
Pass 35'0 36:8 52:8 318 47 42:0 
Fail 205 | 144 | 135 | 232 17 16:9 
No; of atodeng 1000 | тоо'о | тоо'о | 1000 | тоо | xoo'o 
represented 214 299 430 198 83 | 1,224 
Scholars and Non- I "3 | 160o | i59 | 161 | 181r | 162 
scholars combined п $; 3r6 | 25:2 | 2972 | 283 | 289 
In T9 69 67 | 109 3'9 T6 
Pas | 277| 340 | 434 | 285 | 365 | 352 
Fail 15:6 II'$ 8-8 155 13:4 I2'l 
—9—| 
тб: р : : ; , 
Ne ordai ooo | тоо'о | 100-0 | roo'o | тоо'о | 100'0 
represented 404 462 729 366 127 | 2,088 
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П (P) CausRIDGE Unrvensrry TRIPOS Examinations, Parts I AND II 


үн of Class Awatds—Fitsts, Seconds, or Thirds, in the various Triposes (Men 
students only), 


Tripos, Part I 












Classics 
Mathematics 
Modern 
Languages 
Natural 
Sciences 
Mechanical 
Sciences 
Theology 
Geography 












Савт [53:594829 
Clas2 . 309% 29:095 
Class3 . 15:696122:89^ 
No. of 





22:59 21:396 19*996 15:10 1347610876 
46:075 437196 /51:696:43:495327 20 48-676 
31°5%35`6% 48:5 96 41:5 Yol53°9 %0140°6% 


8-4%| 2:976 
$0*096[35:596 
41:69661:695 






































students | 243 | 325 | 302 | 568 | 297 | 53 | 52 | 406 166 | 138 
Average’ 
class. | r62 | 1-74 | 2:09 | z14 | 228 | 2:26 | 2:40 | 2°30 2:33 | 2°58 














Tripos, Part IT 


Class 1. /550%|50°3%|25°4%|31°3% 



































— [191% 83%] — | 48% 
Class 2 . [2927635176 55676 48696 — ]59°5%|35°2%| — |494% 
Class 3. . |15:895|16:696 19:096 20196 — |[r47656596| — |45°8% 
No. of 
students | 120 | 157 | 63 | 179 — |244 | 278 | — 85 
Average І 
cass . | 1-61 | 1°66 | r94 | r89 — | 214 | 2:48 | — | 2:41 











The figures given include all students Scholar, Exhibitioners, and Commoners) for 
the same five colleges and for the eight years on which all the other tables in the text 
ate based. | 

Pass Degrees at Cambridge-—Bot the sake of comparison with provincial universities it 
should be noted that about one-third of the degrees taken at Cambridge are pass 

degrees, almost all being taken by Commoners. 

Figures kindly supplied by the Registrary of Cambridge University show that of 
8,738 B.A, degrees (in Arts and Science) conferred during the seven academic yeats 
1922-1923 to 1928-1929, 5,560 ot 63°7 pet cent, were honours degrees, and 3,178 or 365 
per cent, were ordinary or pass degtees, 
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II (с) CAMBRIDGE Tareos Examinations, PART I. (ALL Sus- 
JECTS.) AVERAGE CLASS AND SUPERIORITY INDICES FOR 
THE Five RESPECTIVE COLLEGES 
































A B c D E Total 
Average Class: 
Scholars . . | rij 140 1'29 ІІ 1'42 1°31 
Exhibitioners $ «| r45 1°65 1:84 1°82 r72 172 
Commoners . ot «a | z42 | 236 | z52 | 236 | 238 2:40 
Superiority Index : 
Scholats/Exhibitioners . | 1-26 | 1-18 143 1°39 I'21 131 
Scholars/Commoners . | 230 1:68 1:95 1:80 r67 1:83 
Exhibitioners/Commoners r6; | r43 1:37 r29 | r38 1:39 
Yat 1922-8 | 1922-9 | 1922--9 | 1922-9 | 1922-9 
Yeats of examinations (9028 192479) ruere [nero | 1922 





Note.—It was no part of the object of this enquiry to consider the relative pet- 
formances of different colleges, Nor must the figures given above be taken as an 
exact representation of the degree of difference between Scholars (etc.) of different 
colleges. To do this we should have to ask whether a larger proportion of 
Scholars in, say, College A took the mathematical or classical tripos (in which firsts 
are frequent) than in other colleges. And analysis did show this was the case for 
Scholars of College A—though not for the Exhibitioners. Again, College B is 
teputed to send a particularly large proportion of its students in for the natural 
sciences tripos, in which, as is shown in the preceding table (Appendix II (b) ), firsts 
are far less frequent than in classics or mathematics, 


П (7) PERFORMANCE IN Parr ll or TRIPOSES TAKEN IN A 
DIFFERENT SUBJECT FROM THAT IN Parr I 





Performance in Part 11 
























































Performance 
in Class 1 Class II Class HI "Total 

Part I = 
Per- Per- Per- Per- 

9. öentaze : centage No. centage No. api 
Class I Scholers . < Ir| 32 [эт | 62 2 6 34 | тоо 
Exhibitioners . 3| до 5| 50 o o то | тоо 
Commoners 2| 20 8| 8o о о 10 | 100 
Total. «41885 33 134] 65 2 4 54 | тоо 
Class П | Scholars . l 3] 19 5| зі 8| 50 16 | roo 
Exhibitioners . | ү 4 |17| 68 7| 28 25 | тоо 
Commoners . || 6 8 145 | 560129 | 36 80 | тоо 
Total 101 8 | 67i 56 | 44| 36 || rar | тоо 
Class Ш | Scholars . -4 of — 2| — о] — 2| — 
Exhibitioners . | o! — oj — + — 1} — 
Commoners 2 2 137] 49 ] 53] 58 92 | 100 
Total. и 2 |39| 41 || 54] 57 95 | 100 














APPENDIX II (её) 191 


П (0) Tse GENERAL ABILITY INVOLVED IN CLASSICS AND 
MATHEMATICS 

One sometimes hears the statement that a man who is good in 
classics ок mathematics can do anything well; and it is some- 
times suggested that he has better all-round ability than those 
who do well in other subjects. 

Although this interesting topic does not come strictly within 
the scope of our enquity, its investigation was taken up, as much 
of the relevant statistical material was already available. 

The table below gives thé later performance of men, who, 
having gained a first class in Part I of the classical or the mathe- 
matical Tripos at Cambridge, subsequently took a different 
subject for their next examination (either a Tripos Part I or 
Part П). For purposes of comparison similar data are also shown 
for those who gained a first class in Part I of other subjects than 
classics or mathematics. 


LATER PERFORMANCE IN A DIFFERENT SUBJECT OF MEN GAINING A First CLASS 
iN Trios Parr I 





Performance in New Subject 





Original Subject in 








Н A Number of Students gaining Average 
hich a First Class 
S was zained ——— C Total Clis 
1st Class 2nd Class 3rd Class gain 
Classics . . | 23 = 36% | 35 = 56% 5 = 8% 63 DI 
Mathematics > ї= 46% | 49 = 45% | 10 = 9% 110 1:63 
5 4 6 
Other subjects . | 25 = 4296 | 31 = 5370 | 3 576 59 | 1563 








Nofe.—'The figures include all the cases in the five selected colleges over а 
period of seven years (viz. 1922-1928, the yzars in which the first Tripos was 
taken), and most of the cases in the other os of the university over a ре оа 
of nine years (1921-1929). The difficulties of tracing the later performance о 
all first-class students in the latter colleges rendered a complete list impossible, 
but those included probably represent more than three-quarters of the total, and 
of course are in no way selected. 


The numbers given above ate not very large, but they suggest 
that a first in classics or mathematics, though almost an invariable 
guarantee of success in a new subject, is no greater guarantee 
than is a first in Triposes other than classics or mathematics. 
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When regard is paid to the time taken in preparation for the 
new subject, the performance of men who took a first in subjects 
other than classics or mathematics is still more meritorious ; for 
while 71 per cent. of such students (in ‘ other’ subjects) took 
their new subject one year after Part I of the Tripos, only 61 per 
cent. of the classics men and no more than то per cent. of the 
mathematics men took only one yeat, the remainder all taking 
two years. 

This superiority cannot be attributed to the taking of Part II 
(rather than Part I) in the new subject by a smaller proportion 
of the ‘ other subjects’? men: for analysis showed that’ while 
65 per cent. of these took a Part IT examination, only 61 per cent. 
of the classics and 41 pet cent. of the mathematics men did so, 
the remainder all taking Part I in their new subject. 

Furthetmote, 75 pet cent. of the mathematics men took as 
their ‘new’ subject either Natural or Mechanical Sciences, 
usually closely allied to the first subject taken. The ‘new’ 
subjects taken by the other students were more varied and would 
usually differ more from the first subject taken. 

Classics and * other subjects? men took a very similar range 
of studies for their second subjects. 
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